Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
2026-05-05T07:53:49Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-05-05T07:53:49Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260505 07:53:49.278791 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
/tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:33621
--webserver_interface=127.0.105.62
--webserver_port=0
--builtin_ntp_servers=127.0.105.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260505 07:53:49.630091 428 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260505 07:53:49.630414 428 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260505 07:53:49.630479 428 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260505 07:53:49.639240 428 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260505 07:53:49.639334 428 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260505 07:53:49.639395 428 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260505 07:53:49.639444 428 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260505 07:53:49.649930 428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.62:33621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.105.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 05 May 2026 07:43:22 UTC on bdcb31816ec0
build id 11754
ASAN enabled
I20260505 07:53:49.651494 428 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260505 07:53:49.653322 428 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260505 07:53:49.661713 433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:49.661921 436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:49.661768 434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:53:49.662918 428 server_base.cc:1061] running on GCE node
I20260505 07:53:49.663918 428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260505 07:53:49.665481 428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260505 07:53:49.666761 428 hybrid_clock.cc:648] HybridClock initialized: now 1777967629666668 us; error 79 us; skew 500 ppm
I20260505 07:53:49.667146 428 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260505 07:53:49.669317 428 webserver.cc:492] Webserver started at http://127.0.105.62:40005/ using document root <none> and password file <none>
I20260505 07:53:49.670050 428 fs_manager.cc:362] Metadata directory not provided
I20260505 07:53:49.670167 428 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260505 07:53:49.670521 428 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260505 07:53:49.673342 428 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/data/instance:
uuid: "379cdfdecc3341ea9d7d9dc09274338d"
format_stamp: "Formatted at 2026-05-05 07:53:49 on dist-test-slave-ksjb"
I20260505 07:53:49.674093 428 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/wal/instance:
uuid: "379cdfdecc3341ea9d7d9dc09274338d"
format_stamp: "Formatted at 2026-05-05 07:53:49 on dist-test-slave-ksjb"
I20260505 07:53:49.679235 428 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260505 07:53:49.682891 442 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260505 07:53:49.684258 428 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260505 07:53:49.684440 428 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/data,/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/wal
uuid: "379cdfdecc3341ea9d7d9dc09274338d"
format_stamp: "Formatted at 2026-05-05 07:53:49 on dist-test-slave-ksjb"
I20260505 07:53:49.684638 428 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260505 07:53:49.703003 428 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260505 07:53:49.704046 428 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260505 07:53:49.704428 428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260505 07:53:49.728065 428 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.62:33621
I20260505 07:53:49.728082 493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.62:33621 every 8 connection(s)
I20260505 07:53:49.729707 428 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/data/info.pb
I20260505 07:53:49.734074 494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:49.736020 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu as pid 428
I20260505 07:53:49.736341 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-0/wal/instance
I20260505 07:53:49.739653 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
/tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:33565
--webserver_interface=127.0.105.61
--webserver_port=0
--builtin_ntp_servers=127.0.105.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260505 07:53:49.742306 494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } has no permanent_uuid. Determining permanent_uuid...
I20260505 07:53:49.758831 494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
W20260505 07:53:49.759991 445 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection refused (error 111)
W20260505 07:53:49.761576 494 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:33565: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection refused (error 111)
I20260505 07:53:49.810952 494 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } attempt: 1
W20260505 07:53:49.813493 494 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:33565: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection refused (error 111)
I20260505 07:53:49.881901 494 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } attempt: 2
W20260505 07:53:49.883858 494 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:33565: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection refused (error 111)
I20260505 07:53:49.975124 494 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } attempt: 3
W20260505 07:53:49.977065 494 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:33565: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection refused (error 111)
W20260505 07:53:50.078100 496 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260505 07:53:50.078380 496 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260505 07:53:50.078452 496 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260505 07:53:50.086696 496 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260505 07:53:50.086829 496 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260505 07:53:50.086905 496 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260505 07:53:50.086961 496 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260505 07:53:50.097325 496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:33565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.105.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 05 May 2026 07:43:22 UTC on bdcb31816ec0
build id 11754
ASAN enabled
I20260505 07:53:50.098832 496 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260505 07:53:50.100522 496 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260505 07:53:50.108671 504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:50.108785 505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:50.109509 507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:53:50.110085 496 server_base.cc:1061] running on GCE node
I20260505 07:53:50.110754 496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260505 07:53:50.112094 496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260505 07:53:50.113317 496 hybrid_clock.cc:648] HybridClock initialized: now 1777967630113288 us; error 41 us; skew 500 ppm
I20260505 07:53:50.113688 496 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260505 07:53:50.115744 496 webserver.cc:492] Webserver started at http://127.0.105.61:43503/ using document root <none> and password file <none>
I20260505 07:53:50.116464 496 fs_manager.cc:362] Metadata directory not provided
I20260505 07:53:50.116600 496 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260505 07:53:50.116947 496 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260505 07:53:50.119200 496 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data/instance:
uuid: "23c57a2c927c4c54bccecfeb3a775311"
format_stamp: "Formatted at 2026-05-05 07:53:50 on dist-test-slave-ksjb"
I20260505 07:53:50.120003 496 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal/instance:
uuid: "23c57a2c927c4c54bccecfeb3a775311"
format_stamp: "Formatted at 2026-05-05 07:53:50 on dist-test-slave-ksjb"
I20260505 07:53:50.120304 494 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } attempt: 4
W20260505 07:53:50.122057 494 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.61:33565: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection refused (error 111)
I20260505 07:53:50.125666 496 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260505 07:53:50.128841 513 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260505 07:53:50.130115 496 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260505 07:53:50.130841 496 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data,/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal
uuid: "23c57a2c927c4c54bccecfeb3a775311"
format_stamp: "Formatted at 2026-05-05 07:53:50 on dist-test-slave-ksjb"
I20260505 07:53:50.131073 496 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260505 07:53:50.147164 496 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260505 07:53:50.148006 496 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260505 07:53:50.148283 496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260505 07:53:50.168671 496 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.61:33565
I20260505 07:53:50.168692 564 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.61:33565 every 8 connection(s)
I20260505 07:53:50.170146 496 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data/info.pb
I20260505 07:53:50.174299 565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:50.178557 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu as pid 496
I20260505 07:53:50.178771 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal/instance
I20260505 07:53:50.180154 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
/tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:35401
--webserver_interface=127.0.105.60
--webserver_port=0
--builtin_ntp_servers=127.0.105.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260505 07:53:50.183233 565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } has no permanent_uuid. Determining permanent_uuid...
I20260505 07:53:50.196753 565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
I20260505 07:53:50.205464 565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } has no permanent_uuid. Determining permanent_uuid...
W20260505 07:53:50.206365 515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
W20260505 07:53:50.208294 565 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:35401: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:53:50.257650 565 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } attempt: 1
W20260505 07:53:50.259505 565 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:35401: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:53:50.327864 565 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } attempt: 2
W20260505 07:53:50.329767 565 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:35401: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:53:50.420990 565 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } attempt: 3
I20260505 07:53:50.421295 494 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } attempt: 5
W20260505 07:53:50.423105 565 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:35401: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:53:50.428269 494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } has no permanent_uuid. Determining permanent_uuid...
W20260505 07:53:50.430107 494 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:35401: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:53:50.481369 494 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } attempt: 1
W20260505 07:53:50.483731 494 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:35401: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
W20260505 07:53:50.513751 567 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260505 07:53:50.514107 567 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260505 07:53:50.514230 567 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260505 07:53:50.523000 567 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260505 07:53:50.523131 567 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260505 07:53:50.523247 567 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260505 07:53:50.523334 567 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260505 07:53:50.534006 567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:35401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.105.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 05 May 2026 07:43:22 UTC on bdcb31816ec0
build id 11754
ASAN enabled
I20260505 07:53:50.535811 567 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260505 07:53:50.537649 567 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260505 07:53:50.546258 576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:50.546135 575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:50.548353 578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:53:50.548395 567 server_base.cc:1061] running on GCE node
I20260505 07:53:50.549194 567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260505 07:53:50.550642 567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260505 07:53:50.551887 567 hybrid_clock.cc:648] HybridClock initialized: now 1777967630551809 us; error 60 us; skew 500 ppm
I20260505 07:53:50.551941 494 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } attempt: 2
I20260505 07:53:50.552317 567 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260505 07:53:50.553723 494 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:35401: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:53:50.554517 567 webserver.cc:492] Webserver started at http://127.0.105.60:44877/ using document root <none> and password file <none>
I20260505 07:53:50.555258 567 fs_manager.cc:362] Metadata directory not provided
I20260505 07:53:50.555395 567 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260505 07:53:50.555806 567 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260505 07:53:50.558323 567 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data/instance:
uuid: "28d3eda04f9c47ae976d19c218b6dcc5"
format_stamp: "Formatted at 2026-05-05 07:53:50 on dist-test-slave-ksjb"
I20260505 07:53:50.559078 567 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal/instance:
uuid: "28d3eda04f9c47ae976d19c218b6dcc5"
format_stamp: "Formatted at 2026-05-05 07:53:50 on dist-test-slave-ksjb"
I20260505 07:53:50.564177 567 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260505 07:53:50.566358 565 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } attempt: 4
I20260505 07:53:50.567780 584 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260505 07:53:50.568883 565 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.0.105.60:35401: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:53:50.569625 567 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260505 07:53:50.569758 567 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data,/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
uuid: "28d3eda04f9c47ae976d19c218b6dcc5"
format_stamp: "Formatted at 2026-05-05 07:53:50 on dist-test-slave-ksjb"
I20260505 07:53:50.569921 567 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260505 07:53:50.591425 567 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260505 07:53:50.592298 567 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260505 07:53:50.592530 567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260505 07:53:50.613655 567 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.60:35401
I20260505 07:53:50.613677 635 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.60:35401 every 8 connection(s)
I20260505 07:53:50.615180 567 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data/info.pb
I20260505 07:53:50.618304 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu as pid 567
I20260505 07:53:50.618479 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal/instance
I20260505 07:53:50.619675 636 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:50.627117 636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } has no permanent_uuid. Determining permanent_uuid...
I20260505 07:53:50.639772 636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } has no permanent_uuid. Determining permanent_uuid...
I20260505 07:53:50.646816 636 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } has no permanent_uuid. Determining permanent_uuid...
I20260505 07:53:50.658773 636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Bootstrap starting.
I20260505 07:53:50.659938 494 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } attempt: 3
I20260505 07:53:50.663321 636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:50.664743 636 log.cc:826] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Log is configured to *not* fsync() on all Append() calls
I20260505 07:53:50.667822 636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: No bootstrap required, opened a new log
I20260505 07:53:50.670722 494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d: Bootstrap starting.
I20260505 07:53:50.673368 636 raft_consensus.cc:359] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:50.673630 636 raft_consensus.cc:385] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:50.673763 636 raft_consensus.cc:740] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d3eda04f9c47ae976d19c218b6dcc5, State: Initialized, Role: FOLLOWER
I20260505 07:53:50.674422 494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:50.674458 636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:50.675858 494 log.cc:826] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d: Log is configured to *not* fsync() on all Append() calls
I20260505 07:53:50.675946 645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:50.676205 645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:53:50.677006 636 sys_catalog.cc:565] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260505 07:53:50.678279 494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d: No bootstrap required, opened a new log
I20260505 07:53:50.683519 645 raft_consensus.cc:493] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:50.683768 645 raft_consensus.cc:515] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:50.684302 494 raft_consensus.cc:359] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:50.684626 494 raft_consensus.cc:385] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:50.684746 494 raft_consensus.cc:740] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 379cdfdecc3341ea9d7d9dc09274338d, State: Initialized, Role: FOLLOWER
I20260505 07:53:50.685699 494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:50.686517 645 leader_election.cc:290] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 379cdfdecc3341ea9d7d9dc09274338d (127.0.105.62:33621), 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565)
I20260505 07:53:50.687032 658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:50.687505 658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:53:50.688369 494 sys_catalog.cc:565] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: configured and running, proceeding with master startup.
I20260505 07:53:50.688400 469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379cdfdecc3341ea9d7d9dc09274338d" is_pre_election: true
I20260505 07:53:50.689482 469 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28d3eda04f9c47ae976d19c218b6dcc5 in term 0.
I20260505 07:53:50.690337 586 leader_election.cc:304] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d3eda04f9c47ae976d19c218b6dcc5, 379cdfdecc3341ea9d7d9dc09274338d; no voters:
I20260505 07:53:50.690766 645 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:50.691008 645 raft_consensus.cc:493] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:50.691143 645 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:53:50.693235 657 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260505 07:53:50.694269 657 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260505 07:53:50.685741 567 master_runner.cc:186] Error getting master registration for 127.0.105.62:33621: OK, instance_id { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" instance_seqno: 1777967629721516 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260505 07:53:50.698566 567 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260505 07:53:50.699925 645 raft_consensus.cc:515] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:50.701205 645 leader_election.cc:290] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [CANDIDATE]: Term 1 election: Requested vote from peers 379cdfdecc3341ea9d7d9dc09274338d (127.0.105.62:33621), 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565)
I20260505 07:53:50.701853 469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "379cdfdecc3341ea9d7d9dc09274338d"
I20260505 07:53:50.702059 469 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:50.705689 469 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d3eda04f9c47ae976d19c218b6dcc5 in term 1.
I20260505 07:53:50.706137 586 leader_election.cc:304] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28d3eda04f9c47ae976d19c218b6dcc5, 379cdfdecc3341ea9d7d9dc09274338d; no voters:
I20260505 07:53:50.706755 645 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 1 FOLLOWER]: Leader election won for term 1
W20260505 07:53:50.707103 448 tablet.cc:2404] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260505 07:53:50.703553 428 master_runner.cc:186] Error getting master registration for 127.0.105.61:33565: OK, instance_id { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" instance_seqno: 1777967630163172 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260505 07:53:50.710386 428 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260505 07:53:50.723356 670 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260505 07:53:50.723922 670 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260505 07:53:50.727809 645 raft_consensus.cc:697] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 1 LEADER]: Becoming Leader. State: Replica: 28d3eda04f9c47ae976d19c218b6dcc5, State: Running, Role: LEADER
I20260505 07:53:50.728420 645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:50.733615 672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d3eda04f9c47ae976d19c218b6dcc5. Latest consensus state: current_term: 1 leader_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:50.733948 672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: This master's current role is: LEADER
I20260505 07:53:50.747042 674 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260505 07:53:50.749320 674 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260505 07:53:50.780380 469 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Refusing update from remote peer 28d3eda04f9c47ae976d19c218b6dcc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260505 07:53:50.781397 645 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:53:50.797778 658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d3eda04f9c47ae976d19c218b6dcc5. Latest consensus state: current_term: 1 leader_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:50.798053 658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:53:50.798878 675 mvcc.cc:204] Tried to move back new op lower bound from 7282555415663124480 to 7282555415468998656. Current Snapshot: MvccSnapshot[applied={T|T < 7282555415663124480}]
I20260505 07:53:50.803025 676 mvcc.cc:204] Tried to move back new op lower bound from 7282555415663124480 to 7282555415468998656. Current Snapshot: MvccSnapshot[applied={T|T < 7282555415663124480}]
I20260505 07:53:50.809460 645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:50.809728 645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: This master's current role is: LEADER
I20260505 07:53:50.815414 658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:50.815650 658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:53:50.820320 674 catalog_manager.cc:1357] Generated new cluster ID: 9b9d7d3d45334a3e97a4e71f5c281d3c
I20260505 07:53:50.820429 674 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260505 07:53:50.856359 674 catalog_manager.cc:1380] Generated new certificate authority record
I20260505 07:53:50.858083 674 catalog_manager.cc:1514] Loading token signing keys...
I20260505 07:53:50.868057 565 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } attempt: 5
I20260505 07:53:50.882521 565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Bootstrap starting.
I20260505 07:53:50.886809 565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:50.889223 565 log.cc:826] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Log is configured to *not* fsync() on all Append() calls
I20260505 07:53:50.891950 674 catalog_manager.cc:6044] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Generated new TSK 0
I20260505 07:53:50.894040 674 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260505 07:53:50.915350 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
/tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.105.1:0
--local_ip_for_outbound_sockets=127.0.105.1
--webserver_interface=127.0.105.1
--webserver_port=0
--tserver_master_addrs=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--builtin_ntp_servers=127.0.105.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260505 07:53:50.929039 565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: No bootstrap required, opened a new log
I20260505 07:53:50.945693 565 raft_consensus.cc:359] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:50.946065 565 raft_consensus.cc:385] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:50.946195 565 raft_consensus.cc:740] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c57a2c927c4c54bccecfeb3a775311, State: Initialized, Role: FOLLOWER
I20260505 07:53:50.947548 565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:50.954691 682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:50.957228 682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:53:50.957137 540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c57a2c927c4c54bccecfeb3a775311" is_pre_election: true
I20260505 07:53:50.957089 565 sys_catalog.cc:565] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: configured and running, proceeding with master startup.
I20260505 07:53:50.957664 540 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28d3eda04f9c47ae976d19c218b6dcc5 in term 0.
I20260505 07:53:50.958380 538 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:50.962811 538 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Refusing update from remote peer 28d3eda04f9c47ae976d19c218b6dcc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260505 07:53:50.963905 645 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:53:50.964027 539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23c57a2c927c4c54bccecfeb3a775311"
I20260505 07:53:50.968509 539 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28d3eda04f9c47ae976d19c218b6dcc5 in term 1.
I20260505 07:53:51.021066 689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:51.021420 689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:53:51.021914 682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d3eda04f9c47ae976d19c218b6dcc5. Latest consensus state: current_term: 1 leader_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:51.022114 682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:53:51.049578 698 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260505 07:53:51.358870 645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:51.359140 645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: This master's current role is: LEADER
W20260505 07:53:51.383001 681 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260505 07:53:51.383308 681 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260505 07:53:51.383387 681 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260505 07:53:51.391762 703 raft_consensus.cc:493] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28d3eda04f9c47ae976d19c218b6dcc5)
I20260505 07:53:51.392175 703 raft_consensus.cc:515] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
W20260505 07:53:51.393236 681 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260505 07:53:51.393445 681 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.105.1
I20260505 07:53:51.393579 703 leader_election.cc:290] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565), 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401)
I20260505 07:53:51.394098 538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "23c57a2c927c4c54bccecfeb3a775311" is_pre_election: true
I20260505 07:53:51.394299 611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" is_pre_election: true
I20260505 07:53:51.395109 444 leader_election.cc:304] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 379cdfdecc3341ea9d7d9dc09274338d; no voters: 23c57a2c927c4c54bccecfeb3a775311, 28d3eda04f9c47ae976d19c218b6dcc5
I20260505 07:53:51.395515 703 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:53:51.404706 681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.105.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.0.105.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.105.1
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 05 May 2026 07:43:22 UTC on bdcb31816ec0
build id 11754
ASAN enabled
I20260505 07:53:51.406332 681 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260505 07:53:51.408193 681 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260505 07:53:51.418684 706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:51.418720 705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:53:51.419934 681 server_base.cc:1061] running on GCE node
W20260505 07:53:51.420642 708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:53:51.421257 681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260505 07:53:51.422187 681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260505 07:53:51.423417 681 hybrid_clock.cc:648] HybridClock initialized: now 1777967631423369 us; error 62 us; skew 500 ppm
I20260505 07:53:51.423774 681 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260505 07:53:51.425824 681 webserver.cc:492] Webserver started at http://127.0.105.1:42019/ using document root <none> and password file <none>
I20260505 07:53:51.426576 681 fs_manager.cc:362] Metadata directory not provided
I20260505 07:53:51.426710 681 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260505 07:53:51.427073 681 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260505 07:53:51.429497 681 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/data/instance:
uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
format_stamp: "Formatted at 2026-05-05 07:53:51 on dist-test-slave-ksjb"
I20260505 07:53:51.430269 681 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/instance:
uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
format_stamp: "Formatted at 2026-05-05 07:53:51 on dist-test-slave-ksjb"
I20260505 07:53:51.435526 681 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260505 07:53:51.439373 714 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260505 07:53:51.440742 681 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260505 07:53:51.440912 681 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal
uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
format_stamp: "Formatted at 2026-05-05 07:53:51 on dist-test-slave-ksjb"
I20260505 07:53:51.441114 681 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260505 07:53:51.523444 681 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260505 07:53:51.524389 681 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260505 07:53:51.525136 681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260505 07:53:51.526108 681 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260505 07:53:51.527832 681 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260505 07:53:51.527947 681 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260505 07:53:51.528080 681 ts_tablet_manager.cc:616] Registered 0 tablets
I20260505 07:53:51.528167 681 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260505 07:53:51.565260 681 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.1:36247
I20260505 07:53:51.565300 826 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.1:36247 every 8 connection(s)
I20260505 07:53:51.566699 681 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/data/info.pb
I20260505 07:53:51.572175 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu as pid 681
I20260505 07:53:51.572443 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/instance
I20260505 07:53:51.579769 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
/tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.105.2:0
--local_ip_for_outbound_sockets=127.0.105.2
--webserver_interface=127.0.105.2
--webserver_port=0
--tserver_master_addrs=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--builtin_ntp_servers=127.0.105.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260505 07:53:51.600911 828 heartbeater.cc:344] Connected to a master server at 127.0.105.62:33621
I20260505 07:53:51.600953 829 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33565
I20260505 07:53:51.601359 829 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:51.601366 828 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:51.602584 829 heartbeater.cc:507] Master 127.0.105.61:33565 requested a full tablet report, sending...
I20260505 07:53:51.602605 828 heartbeater.cc:507] Master 127.0.105.62:33621 requested a full tablet report, sending...
I20260505 07:53:51.604828 459 ts_manager.cc:194] Registered new tserver with Master: eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:51.606777 530 ts_manager.cc:194] Registered new tserver with Master: eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:51.608067 827 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:53:51.608202 827 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:51.608609 827 heartbeater.cc:507] Master 127.0.105.60:35401 requested a full tablet report, sending...
I20260505 07:53:51.610096 601 ts_manager.cc:194] Registered new tserver with Master: eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:51.612185 601 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.105.1:38799
I20260505 07:53:51.726069 670 catalog_manager.cc:1269] Loaded cluster ID: 9b9d7d3d45334a3e97a4e71f5c281d3c
I20260505 07:53:51.726215 670 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d: loading cluster ID for follower catalog manager: success
I20260505 07:53:51.729230 670 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d: acquiring CA information for follower catalog manager: success
I20260505 07:53:51.731016 670 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260505 07:53:51.860668 838 raft_consensus.cc:493] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28d3eda04f9c47ae976d19c218b6dcc5)
I20260505 07:53:51.860886 838 raft_consensus.cc:515] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:51.862254 838 leader_election.cc:290] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 379cdfdecc3341ea9d7d9dc09274338d (127.0.105.62:33621), 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401)
I20260505 07:53:51.862884 468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23c57a2c927c4c54bccecfeb3a775311" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "379cdfdecc3341ea9d7d9dc09274338d" is_pre_election: true
I20260505 07:53:51.862895 611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23c57a2c927c4c54bccecfeb3a775311" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" is_pre_election: true
I20260505 07:53:51.863628 515 leader_election.cc:304] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23c57a2c927c4c54bccecfeb3a775311; no voters: 28d3eda04f9c47ae976d19c218b6dcc5, 379cdfdecc3341ea9d7d9dc09274338d
I20260505 07:53:51.863989 838 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:53:51.937003 835 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260505 07:53:51.937283 835 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260505 07:53:51.937361 835 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260505 07:53:51.945920 835 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260505 07:53:51.946169 835 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.105.2
I20260505 07:53:51.950508 842 raft_consensus.cc:493] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28d3eda04f9c47ae976d19c218b6dcc5)
I20260505 07:53:51.951213 842 raft_consensus.cc:515] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:51.951989 842 leader_election.cc:290] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565), 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401)
I20260505 07:53:51.952492 611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" is_pre_election: true
I20260505 07:53:51.952508 538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "23c57a2c927c4c54bccecfeb3a775311" is_pre_election: true
I20260505 07:53:51.953068 445 leader_election.cc:304] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 379cdfdecc3341ea9d7d9dc09274338d; no voters: 23c57a2c927c4c54bccecfeb3a775311, 28d3eda04f9c47ae976d19c218b6dcc5
I20260505 07:53:51.953358 842 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:53:51.957551 835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.105.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.0.105.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.105.2
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 05 May 2026 07:43:22 UTC on bdcb31816ec0
build id 11754
ASAN enabled
I20260505 07:53:51.959199 835 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260505 07:53:51.961036 835 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260505 07:53:51.970825 845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:51.971010 844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:51.971387 847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:53:51.971735 835 server_base.cc:1061] running on GCE node
I20260505 07:53:51.972347 835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260505 07:53:51.973209 835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260505 07:53:51.974442 835 hybrid_clock.cc:648] HybridClock initialized: now 1777967631974401 us; error 61 us; skew 500 ppm
I20260505 07:53:51.974747 835 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260505 07:53:51.976835 835 webserver.cc:492] Webserver started at http://127.0.105.2:42589/ using document root <none> and password file <none>
I20260505 07:53:51.977559 835 fs_manager.cc:362] Metadata directory not provided
I20260505 07:53:51.977689 835 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260505 07:53:51.978036 835 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260505 07:53:51.980448 835 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/data/instance:
uuid: "d2799f5c6b4f4336b976ca9bbb458533"
format_stamp: "Formatted at 2026-05-05 07:53:51 on dist-test-slave-ksjb"
I20260505 07:53:51.981201 835 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/instance:
uuid: "d2799f5c6b4f4336b976ca9bbb458533"
format_stamp: "Formatted at 2026-05-05 07:53:51 on dist-test-slave-ksjb"
I20260505 07:53:51.986106 835 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260505 07:53:51.989936 853 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260505 07:53:51.991410 835 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260505 07:53:51.991616 835 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal
uuid: "d2799f5c6b4f4336b976ca9bbb458533"
format_stamp: "Formatted at 2026-05-05 07:53:51 on dist-test-slave-ksjb"
I20260505 07:53:51.991827 835 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260505 07:53:52.011085 835 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260505 07:53:52.012022 835 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260505 07:53:52.012295 835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260505 07:53:52.013196 835 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260505 07:53:52.015304 835 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260505 07:53:52.015426 835 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260505 07:53:52.015568 835 ts_tablet_manager.cc:616] Registered 0 tablets
I20260505 07:53:52.015683 835 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260505 07:53:52.052114 835 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.2:43839
I20260505 07:53:52.052163 965 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.2:43839 every 8 connection(s)
I20260505 07:53:52.053901 835 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/data/info.pb
I20260505 07:53:52.056092 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu as pid 835
I20260505 07:53:52.056311 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/instance
I20260505 07:53:52.058959 698 catalog_manager.cc:1269] Loaded cluster ID: 9b9d7d3d45334a3e97a4e71f5c281d3c
I20260505 07:53:52.059170 698 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: loading cluster ID for follower catalog manager: success
I20260505 07:53:52.062583 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
/tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.0.105.3:0
--local_ip_for_outbound_sockets=127.0.105.3
--webserver_interface=127.0.105.3
--webserver_port=0
--tserver_master_addrs=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--builtin_ntp_servers=127.0.105.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260505 07:53:52.072809 698 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: acquiring CA information for follower catalog manager: success
I20260505 07:53:52.076154 698 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260505 07:53:52.099568 968 heartbeater.cc:344] Connected to a master server at 127.0.105.62:33621
I20260505 07:53:52.099970 968 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:52.101056 968 heartbeater.cc:507] Master 127.0.105.62:33621 requested a full tablet report, sending...
I20260505 07:53:52.103183 459 ts_manager.cc:194] Registered new tserver with Master: d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:52.113963 967 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:53:52.114084 967 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:52.114553 967 heartbeater.cc:507] Master 127.0.105.60:35401 requested a full tablet report, sending...
I20260505 07:53:52.115128 827 heartbeater.cc:499] Master 127.0.105.60:35401 was elected leader, sending a full tablet report...
I20260505 07:53:52.115398 971 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33565
I20260505 07:53:52.115513 971 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:52.115566 601 ts_manager.cc:194] Registered new tserver with Master: d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:52.116031 971 heartbeater.cc:507] Master 127.0.105.61:33565 requested a full tablet report, sending...
I20260505 07:53:52.116606 601 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.105.2:35381
I20260505 07:53:52.117592 530 ts_manager.cc:194] Registered new tserver with Master: d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:53:52.427558 970 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260505 07:53:52.427940 970 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260505 07:53:52.428074 970 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260505 07:53:52.436225 970 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260505 07:53:52.436425 970 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.105.3
I20260505 07:53:52.445982 970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.0.105.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.0.105.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.0.105.3
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 05 May 2026 07:43:22 UTC on bdcb31816ec0
build id 11754
ASAN enabled
I20260505 07:53:52.447510 970 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260505 07:53:52.449352 970 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20260505 07:53:52.458853 979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:52.458879 978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260505 07:53:52.459820 981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:53:52.460016 970 server_base.cc:1061] running on GCE node
I20260505 07:53:52.460520 970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260505 07:53:52.461347 970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260505 07:53:52.462625 970 hybrid_clock.cc:648] HybridClock initialized: now 1777967632462568 us; error 52 us; skew 500 ppm
I20260505 07:53:52.462926 970 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260505 07:53:52.465046 970 webserver.cc:492] Webserver started at http://127.0.105.3:32837/ using document root <none> and password file <none>
I20260505 07:53:52.465767 970 fs_manager.cc:362] Metadata directory not provided
I20260505 07:53:52.465889 970 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260505 07:53:52.466202 970 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260505 07:53:52.468497 970 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/data/instance:
uuid: "d63a3e535f8e4dd4a88e447253405b77"
format_stamp: "Formatted at 2026-05-05 07:53:52 on dist-test-slave-ksjb"
I20260505 07:53:52.469750 970 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/instance:
uuid: "d63a3e535f8e4dd4a88e447253405b77"
format_stamp: "Formatted at 2026-05-05 07:53:52 on dist-test-slave-ksjb"
I20260505 07:53:52.474295 970 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260505 07:53:52.477250 987 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260505 07:53:52.478456 970 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260505 07:53:52.478607 970 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal
uuid: "d63a3e535f8e4dd4a88e447253405b77"
format_stamp: "Formatted at 2026-05-05 07:53:52 on dist-test-slave-ksjb"
I20260505 07:53:52.478791 970 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260505 07:53:52.501729 970 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260505 07:53:52.502444 970 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260505 07:53:52.502703 970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260505 07:53:52.503502 970 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260505 07:53:52.505074 970 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260505 07:53:52.505195 970 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260505 07:53:52.505314 970 ts_tablet_manager.cc:616] Registered 0 tablets
I20260505 07:53:52.505401 970 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260505 07:53:52.541045 970 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.3:42069
I20260505 07:53:52.541127 1099 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.3:42069 every 8 connection(s)
I20260505 07:53:52.542338 970 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/data/info.pb
I20260505 07:53:52.544044 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu as pid 970
I20260505 07:53:52.544262 420 external_mini_cluster.cc:1442] Reading /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/instance
I20260505 07:53:52.560431 1100 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:53:52.560925 1100 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:52.562042 1100 heartbeater.cc:507] Master 127.0.105.60:35401 requested a full tablet report, sending...
I20260505 07:53:52.562700 1101 heartbeater.cc:344] Connected to a master server at 127.0.105.62:33621
I20260505 07:53:52.562865 1101 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:52.563426 1101 heartbeater.cc:507] Master 127.0.105.62:33621 requested a full tablet report, sending...
I20260505 07:53:52.563787 600 ts_manager.cc:194] Registered new tserver with Master: d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:52.564545 459 ts_manager.cc:194] Registered new tserver with Master: d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:52.564916 600 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.105.3:35889
I20260505 07:53:52.565444 1102 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33565
I20260505 07:53:52.565582 1102 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:52.566038 1102 heartbeater.cc:507] Master 127.0.105.61:33565 requested a full tablet report, sending...
I20260505 07:53:52.567539 530 ts_manager.cc:194] Registered new tserver with Master: d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:52.574455 420 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20260505 07:53:52.604465 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_0eea08e8b83741208451d0a496b53a5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:52.604557 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_e35515be559148f38af240159406f14c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:52.605062 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_cfb492df001448e6a16c6b1775bdf6b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:52.604485 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_4e7eb11e669540e3a76faba18485739b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:52.604636 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1aafab3816a44373baa97163817f98c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.607089 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e35515be559148f38af240159406f14c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:52.607082 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cfb492df001448e6a16c6b1775bdf6b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:52.607388 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0eea08e8b83741208451d0a496b53a5c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.607352 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_3bef2bc26baf4ace999bdf62c87918e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.608036 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3bef2bc26baf4ace999bdf62c87918e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:52.609200 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1aafab3816a44373baa97163817f98c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:52.607080 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e7eb11e669540e3a76faba18485739b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.619127 967 heartbeater.cc:499] Master 127.0.105.60:35401 was elected leader, sending a full tablet report...
I20260505 07:53:52.622052 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_cd6f1e7c73784523be3f560bd458d7b2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:52.622224 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_8b2e2e36529648cebd2bbdeda19b23b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.622862 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8b2e2e36529648cebd2bbdeda19b23b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:52.622887 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd6f1e7c73784523be3f560bd458d7b2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.624857 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1cae231a7d5647bf887cd8dd8bb0ded9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.625396 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cae231a7d5647bf887cd8dd8bb0ded9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.626343 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_8c90ac7761034713834e4430232a973e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.626914 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c90ac7761034713834e4430232a973e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.691215 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4e7eb11e669540e3a76faba18485739b" } new_table_name: "default.Table_904b36f3c6f94722ad7ac96605e820d8" modify_external_catalogs: true
I20260505 07:53:52.692003 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1aafab3816a44373baa97163817f98c5" } new_table_name: "default.Table_9bff07ce7e6e4ef6b5c4e2de3624cf66" modify_external_catalogs: true
I20260505 07:53:52.688938 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1cae231a7d5647bf887cd8dd8bb0ded9" } modify_external_catalogs: true
I20260505 07:53:52.690601 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0eea08e8b83741208451d0a496b53a5c" } modify_external_catalogs: true
I20260505 07:53:52.693975 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_e35515be559148f38af240159406f14c" } new_table_name: "default.Table_7fe2d440ec38459b9ea99b1471afc296" modify_external_catalogs: true
I20260505 07:53:52.696368 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_cfb492df001448e6a16c6b1775bdf6b9" } new_table_name: "default.Table_45141eb4b8e54de989ac23f6d5613237" modify_external_catalogs: true
I20260505 07:53:52.696890 593 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_3bef2bc26baf4ace999bdf62c87918e4" } new_table_name: "default.Table_2ef173a1b20247b98b2eebee14d30612" modify_external_catalogs: true
I20260505 07:53:52.698379 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8c90ac7761034713834e4430232a973e" } modify_external_catalogs: true
I20260505 07:53:52.698536 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8c90ac7761034713834e4430232a973e" } modify_external_catalogs: true
I20260505 07:53:52.692490 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8b2e2e36529648cebd2bbdeda19b23b6" } modify_external_catalogs: true
I20260505 07:53:52.699448 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0eea08e8b83741208451d0a496b53a5c" } modify_external_catalogs: true
I20260505 07:53:52.700410 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8b2e2e36529648cebd2bbdeda19b23b6" } modify_external_catalogs: true
I20260505 07:53:52.697952 599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_cd6f1e7c73784523be3f560bd458d7b2" } modify_external_catalogs: true
I20260505 07:53:52.704025 599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_cd6f1e7c73784523be3f560bd458d7b2" } modify_external_catalogs: true
I20260505 07:53:52.699908 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1cae231a7d5647bf887cd8dd8bb0ded9" } modify_external_catalogs: true
W20260505 07:53:52.708462 593 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b2f4aff8bf0a43629e43305020df7fa3 has no consensus state
W20260505 07:53:52.711292 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f296602a34634c649d2573cb178b7053 has no consensus state
W20260505 07:53:52.714982 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 56a783ccd69d453ea7e7c004bf55170e has no consensus state
W20260505 07:53:52.716612 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a2b859a5dd54bc8b36f3197b58afcea has no consensus state
W20260505 07:53:52.717442 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c70dedbd58ca402daf0fdde68cc5b574 has no consensus state
W20260505 07:53:52.741241 586 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet b2f4aff8bf0a43629e43305020df7fa3 (table default.Table_2ef173a1b20247b98b2eebee14d30612 [id=d524458d873a48cb980e2f85490232d6], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b2f4aff8bf0a43629e43305020df7fa3 consensus state has no leader
I20260505 07:53:52.753086 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 070d1b1b30d447539911349841638c12
I20260505 07:53:52.754237 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 0fd990e80a90418dbb9b94cb6e2ee156
I20260505 07:53:52.756381 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu pid 496
I20260505 07:53:52.757932 560 generic_service.cc:196] Checking for leaks (request via RPC)
I20260505 07:53:52.775343 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 070d1b1b30d447539911349841638c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:52.777447 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd990e80a90418dbb9b94cb6e2ee156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:52.778187 895 tablet_service.cc:1511] Processing CreateTablet for tablet f296602a34634c649d2573cb178b7053 (DEFAULT_TABLE table=default.Table_7fe2d440ec38459b9ea99b1471afc296 [id=db183a96c4214053beda00a79bdd6b44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.778729 896 tablet_service.cc:1511] Processing CreateTablet for tablet 56a783ccd69d453ea7e7c004bf55170e (DEFAULT_TABLE table=default.Table_904b36f3c6f94722ad7ac96605e820d8 [id=dda64712b2cd4f1286abaf5c08022180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.778986 898 tablet_service.cc:1511] Processing CreateTablet for tablet e1de414720ea44bca392e13c626284bc (DEFAULT_TABLE table=default.Table_1cae231a7d5647bf887cd8dd8bb0ded9 [id=7ba2bfd0de87417c944ea8a4edf99b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.779711 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f296602a34634c649d2573cb178b7053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.783442 893 tablet_service.cc:1511] Processing CreateTablet for tablet c70dedbd58ca402daf0fdde68cc5b574 (DEFAULT_TABLE table=default.Table_9bff07ce7e6e4ef6b5c4e2de3624cf66 [id=b75fe4dac469427c8cf8ccd447f70973]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.783998 899 tablet_service.cc:1511] Processing CreateTablet for tablet 0fd990e80a90418dbb9b94cb6e2ee156 (DEFAULT_TABLE table=default.Table_0eea08e8b83741208451d0a496b53a5c [id=40e8397fa22f44ecb51f34f6f2ac2408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.784479 891 tablet_service.cc:1511] Processing CreateTablet for tablet b705bde46048477b92436eea59cde986 (DEFAULT_TABLE table=default.Table_cd6f1e7c73784523be3f560bd458d7b2 [id=caadebc30e5a4c39877cd29df1e35306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.785166 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1de414720ea44bca392e13c626284bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.786772 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c70dedbd58ca402daf0fdde68cc5b574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.787915 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fd990e80a90418dbb9b94cb6e2ee156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.789042 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b705bde46048477b92436eea59cde986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.790205 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a783ccd69d453ea7e7c004bf55170e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.778034 890 tablet_service.cc:1511] Processing CreateTablet for tablet b2f4aff8bf0a43629e43305020df7fa3 (DEFAULT_TABLE table=default.Table_2ef173a1b20247b98b2eebee14d30612 [id=d524458d873a48cb980e2f85490232d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.791357 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2f4aff8bf0a43629e43305020df7fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.792167 897 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2b859a5dd54bc8b36f3197b58afcea (DEFAULT_TABLE table=default.Table_45141eb4b8e54de989ac23f6d5613237 [id=b8e0010041cc477d9ce226cca5a28a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.792763 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2b859a5dd54bc8b36f3197b58afcea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.793485 900 tablet_service.cc:1511] Processing CreateTablet for tablet 689792d9ea734e0bbf1df1c74dab6f83 (DEFAULT_TABLE table=default.Table_8b2e2e36529648cebd2bbdeda19b23b6 [id=0407ce7e0583423e92f51c513897877d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.794044 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 689792d9ea734e0bbf1df1c74dab6f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.795126 901 tablet_service.cc:1511] Processing CreateTablet for tablet 070d1b1b30d447539911349841638c12 (DEFAULT_TABLE table=default.Table_8c90ac7761034713834e4430232a973e [id=9fcb3767fb974e7fba21836bf2a06dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.795763 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 070d1b1b30d447539911349841638c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.882308 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_23d3119ff64f4b9caf3ce96d7520fccf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.882907 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_23d3119ff64f4b9caf3ce96d7520fccf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.884202 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_53ef916ef6d94510862d6efec4f98487"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.884678 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53ef916ef6d94510862d6efec4f98487 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.886085 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1685a2ce9c564394b1fcb34a609ec293"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.886529 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1685a2ce9c564394b1fcb34a609ec293 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.886689 1172 tablet_bootstrap.cc:492] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:52.887151 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_448efd98249942379556e125b25d169c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.887640 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_448efd98249942379556e125b25d169c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.887554 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_26fac172e5aa414db72cbf7d03d99d8f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:52.887830 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.888082 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_26fac172e5aa414db72cbf7d03d99d8f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:52.888233 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.889159 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_37f54e4b91df4ca9bcbe16c5b52f6ba8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.889566 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37f54e4b91df4ca9bcbe16c5b52f6ba8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.892429 1172 tablet_bootstrap.cc:654] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:52.893098 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet b705bde46048477b92436eea59cde986
I20260505 07:53:52.896801 1172 log.cc:826] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: Log is configured to *not* fsync() on all Append() calls
I20260505 07:53:52.902137 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7f23cbfec9794a2bbc9e9dad2d07011c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.902833 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f23cbfec9794a2bbc9e9dad2d07011c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.905681 760 tablet_service.cc:1511] Processing CreateTablet for tablet 0fd990e80a90418dbb9b94cb6e2ee156 (DEFAULT_TABLE table=default.Table_0eea08e8b83741208451d0a496b53a5c [id=40e8397fa22f44ecb51f34f6f2ac2408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.907055 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fd990e80a90418dbb9b94cb6e2ee156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.917683 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 0fd990e80a90418dbb9b94cb6e2ee156 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:52.917758 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet e1de414720ea44bca392e13c626284bc
I20260505 07:53:52.918238 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 689792d9ea734e0bbf1df1c74dab6f83
I20260505 07:53:52.918457 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 0fd990e80a90418dbb9b94cb6e2ee156 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:52.918498 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 0fd990e80a90418dbb9b94cb6e2ee156 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:52.918954 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_dda9e1fe74264c2eab25cecc65079c93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.919452 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dda9e1fe74264c2eab25cecc65079c93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.919600 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 689792d9ea734e0bbf1df1c74dab6f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:52.920822 757 tablet_service.cc:1511] Processing CreateTablet for tablet 56a783ccd69d453ea7e7c004bf55170e (DEFAULT_TABLE table=default.Table_904b36f3c6f94722ad7ac96605e820d8 [id=dda64712b2cd4f1286abaf5c08022180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.921262 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2ef173a1b20247b98b2eebee14d30612" } new_table_name: "default.Table_5e3e2127a0634fc9a6b3fa6d6523d697" modify_external_catalogs: true
I20260505 07:53:52.921504 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a783ccd69d453ea7e7c004bf55170e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.923933 754 tablet_service.cc:1511] Processing CreateTablet for tablet c70dedbd58ca402daf0fdde68cc5b574 (DEFAULT_TABLE table=default.Table_9bff07ce7e6e4ef6b5c4e2de3624cf66 [id=b75fe4dac469427c8cf8ccd447f70973]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.924571 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c70dedbd58ca402daf0fdde68cc5b574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.918978 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_aa53055d0748464697a41feba3c4f036"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:52.925117 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa53055d0748464697a41feba3c4f036 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:52.925674 758 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2b859a5dd54bc8b36f3197b58afcea (DEFAULT_TABLE table=default.Table_45141eb4b8e54de989ac23f6d5613237 [id=b8e0010041cc477d9ce226cca5a28a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.911274 753 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd990e80a90418dbb9b94cb6e2ee156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:52.920099 755 tablet_service.cc:1558] Processing DeleteTablet for tablet 070d1b1b30d447539911349841638c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:52.928440 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2b859a5dd54bc8b36f3197b58afcea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.931653 752 tablet_service.cc:1511] Processing CreateTablet for tablet b705bde46048477b92436eea59cde986 (DEFAULT_TABLE table=default.Table_cd6f1e7c73784523be3f560bd458d7b2 [id=caadebc30e5a4c39877cd29df1e35306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.932180 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b705bde46048477b92436eea59cde986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.935678 756 tablet_service.cc:1511] Processing CreateTablet for tablet f296602a34634c649d2573cb178b7053 (DEFAULT_TABLE table=default.Table_7fe2d440ec38459b9ea99b1471afc296 [id=db183a96c4214053beda00a79bdd6b44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.936203 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f296602a34634c649d2573cb178b7053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.940183 596 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_904b36f3c6f94722ad7ac96605e820d8" } new_table_name: "default.Table_7cdbf8c75e364774b5e3db4310b3a1e0" modify_external_catalogs: true
I20260505 07:53:52.942097 761 tablet_service.cc:1511] Processing CreateTablet for tablet 689792d9ea734e0bbf1df1c74dab6f83 (DEFAULT_TABLE table=default.Table_8b2e2e36529648cebd2bbdeda19b23b6 [id=0407ce7e0583423e92f51c513897877d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.942682 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 689792d9ea734e0bbf1df1c74dab6f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.947753 759 tablet_service.cc:1511] Processing CreateTablet for tablet e1de414720ea44bca392e13c626284bc (DEFAULT_TABLE table=default.Table_1cae231a7d5647bf887cd8dd8bb0ded9 [id=7ba2bfd0de87417c944ea8a4edf99b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.948304 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1de414720ea44bca392e13c626284bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.960222 751 tablet_service.cc:1511] Processing CreateTablet for tablet b2f4aff8bf0a43629e43305020df7fa3 (DEFAULT_TABLE table=default.Table_2ef173a1b20247b98b2eebee14d30612 [id=d524458d873a48cb980e2f85490232d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.960799 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2f4aff8bf0a43629e43305020df7fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:52.964650 750 tablet_service.cc:1558] Processing DeleteTablet for tablet e1de414720ea44bca392e13c626284bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:52.965206 755 tablet_service.cc:1558] Processing DeleteTablet for tablet b705bde46048477b92436eea59cde986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:52.965493 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 689792d9ea734e0bbf1df1c74dab6f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:52.898479 762 tablet_service.cc:1511] Processing CreateTablet for tablet 070d1b1b30d447539911349841638c12 (DEFAULT_TABLE table=default.Table_8c90ac7761034713834e4430232a973e [id=9fcb3767fb974e7fba21836bf2a06dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:52.978770 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 070d1b1b30d447539911349841638c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.028090 891 tablet_service.cc:1558] Processing DeleteTablet for tablet e1de414720ea44bca392e13c626284bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.030131 890 tablet_service.cc:1558] Processing DeleteTablet for tablet b705bde46048477b92436eea59cde986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.039719 1176 tablet_bootstrap.cc:492] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:53.045059 1176 tablet_bootstrap.cc:654] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.049865 1176 log.cc:826] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: Log is configured to *not* fsync() on all Append() calls
W20260505 07:53:53.052779 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 070d1b1b30d447539911349841638c12 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 070d1b1b30d447539911349841638c12 already in progress: creating tablet
W20260505 07:53:53.054062 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 0fd990e80a90418dbb9b94cb6e2ee156 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fd990e80a90418dbb9b94cb6e2ee156 already in progress: creating tablet
W20260505 07:53:53.055070 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 689792d9ea734e0bbf1df1c74dab6f83 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 689792d9ea734e0bbf1df1c74dab6f83 already in progress: creating tablet
W20260505 07:53:53.055899 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e1de414720ea44bca392e13c626284bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1de414720ea44bca392e13c626284bc already in progress: creating tablet
W20260505 07:53:53.056722 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet b705bde46048477b92436eea59cde986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b705bde46048477b92436eea59cde986 already in progress: creating tablet
I20260505 07:53:53.057616 1176 tablet_bootstrap.cc:492] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:53.057982 1176 ts_tablet_manager.cc:1403] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.000s
I20260505 07:53:53.067941 1100 heartbeater.cc:499] Master 127.0.105.60:35401 was elected leader, sending a full tablet report...
I20260505 07:53:53.068198 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd990e80a90418dbb9b94cb6e2ee156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.069582 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 689792d9ea734e0bbf1df1c74dab6f83 (DEFAULT_TABLE table=default.Table_8b2e2e36529648cebd2bbdeda19b23b6 [id=0407ce7e0583423e92f51c513897877d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.070266 1026 tablet_service.cc:1511] Processing CreateTablet for tablet c70dedbd58ca402daf0fdde68cc5b574 (DEFAULT_TABLE table=default.Table_9bff07ce7e6e4ef6b5c4e2de3624cf66 [id=b75fe4dac469427c8cf8ccd447f70973]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:53.070986 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 0fd990e80a90418dbb9b94cb6e2ee156 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fd990e80a90418dbb9b94cb6e2ee156 already in progress: creating tablet
I20260505 07:53:53.071183 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c70dedbd58ca402daf0fdde68cc5b574. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.068392 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet e1de414720ea44bca392e13c626284bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.072952 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2b859a5dd54bc8b36f3197b58afcea (DEFAULT_TABLE table=default.Table_45141eb4b8e54de989ac23f6d5613237 [id=b8e0010041cc477d9ce226cca5a28a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.073503 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2b859a5dd54bc8b36f3197b58afcea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.074359 1029 tablet_service.cc:1511] Processing CreateTablet for tablet f296602a34634c649d2573cb178b7053 (DEFAULT_TABLE table=default.Table_7fe2d440ec38459b9ea99b1471afc296 [id=db183a96c4214053beda00a79bdd6b44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.074816 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f296602a34634c649d2573cb178b7053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.075676 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 0fd990e80a90418dbb9b94cb6e2ee156 (DEFAULT_TABLE table=default.Table_0eea08e8b83741208451d0a496b53a5c [id=40e8397fa22f44ecb51f34f6f2ac2408]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.076299 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet b705bde46048477b92436eea59cde986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.076609 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 689792d9ea734e0bbf1df1c74dab6f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.076675 1024 tablet_service.cc:1511] Processing CreateTablet for tablet b2f4aff8bf0a43629e43305020df7fa3 (DEFAULT_TABLE table=default.Table_2ef173a1b20247b98b2eebee14d30612 [id=d524458d873a48cb980e2f85490232d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.076618 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fd990e80a90418dbb9b94cb6e2ee156. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.076771 1028 tablet_service.cc:1558] Processing DeleteTablet for tablet 070d1b1b30d447539911349841638c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.077193 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2f4aff8bf0a43629e43305020df7fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.077442 1176 raft_consensus.cc:359] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.077824 1176 raft_consensus.cc:385] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.077912 1176 raft_consensus.cc:740] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.077977 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 689792d9ea734e0bbf1df1c74dab6f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.078526 1032 tablet_service.cc:1511] Processing CreateTablet for tablet e1de414720ea44bca392e13c626284bc (DEFAULT_TABLE table=default.Table_1cae231a7d5647bf887cd8dd8bb0ded9 [id=7ba2bfd0de87417c944ea8a4edf99b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.078557 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 070d1b1b30d447539911349841638c12 (DEFAULT_TABLE table=default.Table_8c90ac7761034713834e4430232a973e [id=9fcb3767fb974e7fba21836bf2a06dc6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.079151 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 56a783ccd69d453ea7e7c004bf55170e (DEFAULT_TABLE table=default.Table_904b36f3c6f94722ad7ac96605e820d8 [id=dda64712b2cd4f1286abaf5c08022180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.079702 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a783ccd69d453ea7e7c004bf55170e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.080407 1172 tablet_bootstrap.cc:492] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:53.080687 1172 ts_tablet_manager.cc:1403] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.194s user 0.056s sys 0.084s
I20260505 07:53:53.083433 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 070d1b1b30d447539911349841638c12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.084342 1025 tablet_service.cc:1511] Processing CreateTablet for tablet b705bde46048477b92436eea59cde986 (DEFAULT_TABLE table=default.Table_cd6f1e7c73784523be3f560bd458d7b2 [id=caadebc30e5a4c39877cd29df1e35306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.084782 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b705bde46048477b92436eea59cde986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.087893 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1de414720ea44bca392e13c626284bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.088272 1172 raft_consensus.cc:359] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.088598 1172 raft_consensus.cc:385] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.088692 1172 raft_consensus.cc:740] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.089202 1176 consensus_queue.cc:260] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.090170 1172 consensus_queue.cc:260] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.102102 1176 ts_tablet_manager.cc:1434] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.044s user 0.012s sys 0.003s
I20260505 07:53:53.096038 1172 ts_tablet_manager.cc:1434] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260505 07:53:53.102653 1172 tablet_bootstrap.cc:492] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:53.105947 1176 tablet_bootstrap.cc:492] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:53.107998 1176 tablet_bootstrap.cc:654] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.110909 1172 tablet_bootstrap.cc:654] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:53.112694 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 070d1b1b30d447539911349841638c12 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 070d1b1b30d447539911349841638c12 already in progress: creating tablet
W20260505 07:53:53.112998 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e1de414720ea44bca392e13c626284bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1de414720ea44bca392e13c626284bc already in progress: creating tablet
W20260505 07:53:53.113188 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet b705bde46048477b92436eea59cde986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b705bde46048477b92436eea59cde986 already in progress: creating tablet
W20260505 07:53:53.113412 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 689792d9ea734e0bbf1df1c74dab6f83 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 689792d9ea734e0bbf1df1c74dab6f83 already in progress: creating tablet
W20260505 07:53:53.114349 973 tablet.cc:2404] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260505 07:53:53.116089 1176 tablet_bootstrap.cc:492] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:53.116235 1176 ts_tablet_manager.cc:1403] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260505 07:53:53.117122 1176 raft_consensus.cc:359] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.117318 1176 raft_consensus.cc:385] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.117403 1176 raft_consensus.cc:740] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.117650 1176 consensus_queue.cc:260] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.119839 1176 ts_tablet_manager.cc:1434] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260505 07:53:53.121206 1176 tablet_bootstrap.cc:492] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:53.123274 1176 tablet_bootstrap.cc:654] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.125406 1172 tablet_bootstrap.cc:492] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:53.125659 1172 ts_tablet_manager.cc:1403] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.023s user 0.005s sys 0.000s
I20260505 07:53:53.126721 1172 raft_consensus.cc:359] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.127038 1172 raft_consensus.cc:385] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.127189 1172 raft_consensus.cc:740] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.127566 1172 consensus_queue.cc:260] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.128310 1172 ts_tablet_manager.cc:1434] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260505 07:53:53.128499 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_45141eb4b8e54de989ac23f6d5613237" } new_table_name: "default.Table_ae1a7c2a1099433289c88a814e0b132e" modify_external_catalogs: true
I20260505 07:53:53.128607 1172 tablet_bootstrap.cc:492] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:53.129890 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7fe2d440ec38459b9ea99b1471afc296" } new_table_name: "default.Table_26dcd25c1a124d12a99d4c5a02bb4223" modify_external_catalogs: true
I20260505 07:53:53.130169 1172 tablet_bootstrap.cc:654] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.131150 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_9bff07ce7e6e4ef6b5c4e2de3624cf66" } new_table_name: "default.Table_2dba7ce60556464488c3d723065cbbfd" modify_external_catalogs: true
I20260505 07:53:53.132642 599 master_service.cc:954] ReplaceTablet: received request to replace tablet b705bde46048477b92436eea59cde986 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.133064 599 master_service.cc:954] ReplaceTablet: received request to replace tablet b2f4aff8bf0a43629e43305020df7fa3 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.133375 1172 tablet_bootstrap.cc:492] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:53.133530 1172 ts_tablet_manager.cc:1403] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260505 07:53:53.134227 1172 raft_consensus.cc:359] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.134341 900 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd990e80a90418dbb9b94cb6e2ee156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.134487 1172 raft_consensus.cc:385] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.134598 1172 raft_consensus.cc:740] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.134685 1171 tablet_replica.cc:333] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:53.134827 1172 consensus_queue.cc:260] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.135133 1171 raft_consensus.cc:2243] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:53.135389 1172 ts_tablet_manager.cc:1434] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:53:53.135484 1171 raft_consensus.cc:2272] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:53.136763 1176 tablet_bootstrap.cc:492] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:53.136920 1176 ts_tablet_manager.cc:1403] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.004s
I20260505 07:53:53.137444 1171 ts_tablet_manager.cc:1916] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:53.137495 1176 raft_consensus.cc:359] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.137625 1176 raft_consensus.cc:385] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.137688 1176 raft_consensus.cc:740] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
W20260505 07:53:53.137689 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 0fd990e80a90418dbb9b94cb6e2ee156 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0fd990e80a90418dbb9b94cb6e2ee156 already in progress: creating tablet
W20260505 07:53:53.137902 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e1de414720ea44bca392e13c626284bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1de414720ea44bca392e13c626284bc already in progress: creating tablet
I20260505 07:53:53.137918 1176 consensus_queue.cc:260] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:53:53.138094 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet b705bde46048477b92436eea59cde986 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b705bde46048477b92436eea59cde986 already in progress: creating tablet
W20260505 07:53:53.138266 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 689792d9ea734e0bbf1df1c74dab6f83 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 689792d9ea734e0bbf1df1c74dab6f83 already in progress: creating tablet
I20260505 07:53:53.138296 1176 ts_tablet_manager.cc:1434] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260505 07:53:53.138460 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 070d1b1b30d447539911349841638c12 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 070d1b1b30d447539911349841638c12 already in progress: creating tablet
I20260505 07:53:53.138502 1176 tablet_bootstrap.cc:492] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:53.140084 1176 tablet_bootstrap.cc:654] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.140290 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_53ef916ef6d94510862d6efec4f98487" } modify_external_catalogs: true
I20260505 07:53:53.140457 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_53ef916ef6d94510862d6efec4f98487" } modify_external_catalogs: true
I20260505 07:53:53.144179 1176 tablet_bootstrap.cc:492] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:53.144325 1176 ts_tablet_manager.cc:1403] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260505 07:53:53.144944 1172 tablet_bootstrap.cc:492] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:53.145052 1176 raft_consensus.cc:359] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.145234 1176 raft_consensus.cc:385] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.145330 1176 raft_consensus.cc:740] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.145680 1176 consensus_queue.cc:260] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.146925 1176 ts_tablet_manager.cc:1434] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:53:53.147186 1176 tablet_bootstrap.cc:492] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:53.148747 1172 tablet_bootstrap.cc:654] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.150183 1176 tablet_bootstrap.cc:654] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.151551 1181 tablet_bootstrap.cc:492] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:53.151605 1171 ts_tablet_manager.cc:1929] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.151750 1171 log.cc:1199] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/0fd990e80a90418dbb9b94cb6e2ee156
I20260505 07:53:53.152166 1171 ts_tablet_manager.cc:1950] T 0fd990e80a90418dbb9b94cb6e2ee156 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:53.153713 594 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b2f4aff8bf0a43629e43305020df7fa3 (table default.Table_2ef173a1b20247b98b2eebee14d30612 [id=d524458d873a48cb980e2f85490232d6]). Expected version 1 got 0
I20260505 07:53:53.154327 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 0fd990e80a90418dbb9b94cb6e2ee156 (table default.Table_0eea08e8b83741208451d0a496b53a5c [id=40e8397fa22f44ecb51f34f6f2ac2408]) successfully deleted
I20260505 07:53:53.155310 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 8087cab2da594607a0c3f7cf5839c6f3 (DEFAULT_TABLE table=default.Table_53ef916ef6d94510862d6efec4f98487 [id=2b0d644a9b4a490685837b5e427cffb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.156189 762 tablet_service.cc:1511] Processing CreateTablet for tablet 8087cab2da594607a0c3f7cf5839c6f3 (DEFAULT_TABLE table=default.Table_53ef916ef6d94510862d6efec4f98487 [id=2b0d644a9b4a490685837b5e427cffb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.156425 900 tablet_service.cc:1511] Processing CreateTablet for tablet a25715f0fa504e92b54c48712c2c2f48 (DEFAULT_TABLE table=default.Table_23d3119ff64f4b9caf3ce96d7520fccf [id=0ac6c7c427534982aa4dca340c2001bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.156553 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8087cab2da594607a0c3f7cf5839c6f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.156545 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8087cab2da594607a0c3f7cf5839c6f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.156777 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a25715f0fa504e92b54c48712c2c2f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.157982 1030 tablet_service.cc:1511] Processing CreateTablet for tablet a25715f0fa504e92b54c48712c2c2f48 (DEFAULT_TABLE table=default.Table_23d3119ff64f4b9caf3ce96d7520fccf [id=0ac6c7c427534982aa4dca340c2001bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.158357 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a25715f0fa504e92b54c48712c2c2f48. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:53.160240 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b2f4aff8bf0a43629e43305020df7fa3 consensus state has no leader
I20260505 07:53:53.160833 895 tablet_service.cc:1511] Processing CreateTablet for tablet 8087cab2da594607a0c3f7cf5839c6f3 (DEFAULT_TABLE table=default.Table_53ef916ef6d94510862d6efec4f98487 [id=2b0d644a9b4a490685837b5e427cffb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.161149 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8087cab2da594607a0c3f7cf5839c6f3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:53.161926 596 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 56a783ccd69d453ea7e7c004bf55170e consensus state has no leader
I20260505 07:53:53.162225 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_e78ac004e1af4644a442dee2c23ac5c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.162660 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e78ac004e1af4644a442dee2c23ac5c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.164026 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6208b89234f342409bfac888cc6cfe94"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.164521 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6208b89234f342409bfac888cc6cfe94 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.165464 594 master_service.cc:954] ReplaceTablet: received request to replace tablet b2f4aff8bf0a43629e43305020df7fa3 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.168968 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_23d3119ff64f4b9caf3ce96d7520fccf" } modify_external_catalogs: true
I20260505 07:53:53.169190 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_23d3119ff64f4b9caf3ce96d7520fccf" } modify_external_catalogs: true
I20260505 07:53:53.171353 753 tablet_service.cc:1511] Processing CreateTablet for tablet a25715f0fa504e92b54c48712c2c2f48 (DEFAULT_TABLE table=default.Table_23d3119ff64f4b9caf3ce96d7520fccf [id=0ac6c7c427534982aa4dca340c2001bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.171752 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a25715f0fa504e92b54c48712c2c2f48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.172152 1181 tablet_bootstrap.cc:654] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.173522 1181 log.cc:826] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: Log is configured to *not* fsync() on all Append() calls
I20260505 07:53:53.183331 1181 tablet_bootstrap.cc:492] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:53.183753 1181 ts_tablet_manager.cc:1403] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.033s user 0.012s sys 0.003s
I20260505 07:53:53.186033 1172 tablet_bootstrap.cc:492] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:53.192059 1181 raft_consensus.cc:359] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.192523 1172 ts_tablet_manager.cc:1403] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.048s user 0.003s sys 0.007s
I20260505 07:53:53.193277 1172 raft_consensus.cc:359] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.193497 1172 raft_consensus.cc:385] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.193624 1172 raft_consensus.cc:740] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.193898 1172 consensus_queue.cc:260] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.194429 1172 ts_tablet_manager.cc:1434] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:53:53.194741 1172 tablet_bootstrap.cc:492] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:53.196347 1172 tablet_bootstrap.cc:654] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.197822 1181 raft_consensus.cc:385] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.198007 1181 raft_consensus.cc:740] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.198917 1181 consensus_queue.cc:260] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.203545 1172 tablet_bootstrap.cc:492] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:53.203729 1172 ts_tablet_manager.cc:1403] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.004s
I20260505 07:53:53.204442 1172 raft_consensus.cc:359] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:53:53.204627 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6a2b859a5dd54bc8b36f3197b58afcea consensus state has no leader
I20260505 07:53:53.204628 1172 raft_consensus.cc:385] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.204704 1172 raft_consensus.cc:740] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.204941 1172 consensus_queue.cc:260] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:53.206053 1172 ts_tablet_manager.cc:1434] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:53:53.206354 1172 tablet_bootstrap.cc:492] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:53.207922 1172 tablet_bootstrap.cc:654] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:53.210884 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f296602a34634c649d2573cb178b7053 consensus state has no leader
I20260505 07:53:53.211244 601 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_26fac172e5aa414db72cbf7d03d99d8f" } modify_external_catalogs: true
I20260505 07:53:53.211408 601 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_26fac172e5aa414db72cbf7d03d99d8f" } modify_external_catalogs: true
W20260505 07:53:53.213069 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c70dedbd58ca402daf0fdde68cc5b574 consensus state has no leader
I20260505 07:53:53.213342 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7f23cbfec9794a2bbc9e9dad2d07011c" } modify_external_catalogs: true
I20260505 07:53:53.213505 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7f23cbfec9794a2bbc9e9dad2d07011c" } modify_external_catalogs: true
I20260505 07:53:53.222704 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet b2f4aff8bf0a43629e43305020df7fa3
I20260505 07:53:53.223167 599 catalog_manager.cc:6741] ReplaceTablet: tablet b2f4aff8bf0a43629e43305020df7fa3 deleted and replaced by tablet 2850a301ac4f435a96742b00bf54baa9
I20260505 07:53:53.223408 599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1685a2ce9c564394b1fcb34a609ec293" } modify_external_catalogs: true
I20260505 07:53:53.223546 900 tablet_service.cc:1558] Processing DeleteTablet for tablet b2f4aff8bf0a43629e43305020df7fa3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2850a301ac4f435a96742b00bf54baa9) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.223568 599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1685a2ce9c564394b1fcb34a609ec293" } modify_external_catalogs: true
I20260505 07:53:53.223839 1171 tablet_replica.cc:333] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:53.224059 1171 raft_consensus.cc:2243] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:53.224225 1171 raft_consensus.cc:2272] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:53.225708 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 0fd990e80a90418dbb9b94cb6e2ee156 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.226076 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 6a2b859a5dd54bc8b36f3197b58afcea from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.226655 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet b2f4aff8bf0a43629e43305020df7fa3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2850a301ac4f435a96742b00bf54baa9) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.227555 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd990e80a90418dbb9b94cb6e2ee156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.227792 762 tablet_service.cc:1558] Processing DeleteTablet for tablet b2f4aff8bf0a43629e43305020df7fa3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2850a301ac4f435a96742b00bf54baa9) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.228160 1175 tablet_replica.cc:333] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:53.228713 1175 raft_consensus.cc:2243] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:53.229121 1175 raft_consensus.cc:2272] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:53.229815 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 8087cab2da594607a0c3f7cf5839c6f3
I20260505 07:53:53.231146 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 8087cab2da594607a0c3f7cf5839c6f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.231968 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 8087cab2da594607a0c3f7cf5839c6f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.232026 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 8087cab2da594607a0c3f7cf5839c6f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.232925 1175 ts_tablet_manager.cc:1916] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:53.234949 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet a25715f0fa504e92b54c48712c2c2f48
I20260505 07:53:53.235231 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 689792d9ea734e0bbf1df1c74dab6f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.235769 895 tablet_service.cc:1558] Processing DeleteTablet for tablet a25715f0fa504e92b54c48712c2c2f48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.235790 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet a25715f0fa504e92b54c48712c2c2f48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.236362 762 tablet_service.cc:1558] Processing DeleteTablet for tablet a25715f0fa504e92b54c48712c2c2f48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.237784 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 070d1b1b30d447539911349841638c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.238744 1181 ts_tablet_manager.cc:1434] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.055s user 0.009s sys 0.011s
I20260505 07:53:53.240948 1181 tablet_bootstrap.cc:492] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:53.242594 1181 tablet_bootstrap.cc:654] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.247049 601 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet cd29ec3c9865453f8dc11b0ee14d517d
I20260505 07:53:53.251396 599 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 8d36206f4daf4e758c2c275dbc2993a4
W20260505 07:53:53.258955 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet b2f4aff8bf0a43629e43305020df7fa3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b2f4aff8bf0a43629e43305020df7fa3 already in progress: creating tablet
W20260505 07:53:53.259202 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8087cab2da594607a0c3f7cf5839c6f3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8087cab2da594607a0c3f7cf5839c6f3 already in progress: creating tablet
I20260505 07:53:53.259281 900 tablet_service.cc:1558] Processing DeleteTablet for tablet b705bde46048477b92436eea59cde986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:53.259821 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet a25715f0fa504e92b54c48712c2c2f48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a25715f0fa504e92b54c48712c2c2f48 already in progress: creating tablet
I20260505 07:53:53.274358 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 31b151ff45804fbaa76cfbec86c6fd7a
I20260505 07:53:53.275244 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.275725 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.276141 595 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 56a783ccd69d453ea7e7c004bf55170e (table default.Table_7cdbf8c75e364774b5e3db4310b3a1e0 [id=dda64712b2cd4f1286abaf5c08022180]). Expected version 2 got 0
I20260505 07:53:53.276664 600 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6a2b859a5dd54bc8b36f3197b58afcea (table default.Table_ae1a7c2a1099433289c88a814e0b132e [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 2 got 0
I20260505 07:53:53.276906 600 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f296602a34634c649d2573cb178b7053 (table default.Table_26dcd25c1a124d12a99d4c5a02bb4223 [id=db183a96c4214053beda00a79bdd6b44]). Expected version 2 got 0
I20260505 07:53:53.278069 657 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 31b151ff45804fbaa76cfbec86c6fd7a
I20260505 07:53:53.281942 595 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 56a783ccd69d453ea7e7c004bf55170e (table default.Table_7cdbf8c75e364774b5e3db4310b3a1e0 [id=dda64712b2cd4f1286abaf5c08022180]). Expected version 2 got 0
W20260505 07:53:53.282919 587 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 31b151ff45804fbaa76cfbec86c6fd7a because the tablet was not found. No further retry: Not found: Tablet not found: 31b151ff45804fbaa76cfbec86c6fd7a
I20260505 07:53:53.284099 762 tablet_service.cc:1558] Processing DeleteTablet for tablet b705bde46048477b92436eea59cde986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.286048 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 6a2b859a5dd54bc8b36f3197b58afcea
I20260505 07:53:53.286568 592 catalog_manager.cc:6741] ReplaceTablet: tablet 6a2b859a5dd54bc8b36f3197b58afcea deleted and replaced by tablet 8ca4cf248adb4f768c5586d72614ad42
I20260505 07:53:53.287926 900 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.288046 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu with pid 496
I20260505 07:53:53.290964 1171 ts_tablet_manager.cc:1916] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:53.291639 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a (DEFAULT_TABLE table=default.Table_7f23cbfec9794a2bbc9e9dad2d07011c [id=b25c09d8fb8347af83c69e76a523867e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.292059 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b151ff45804fbaa76cfbec86c6fd7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.295989 1034 tablet_service.cc:1511] Processing CreateTablet for tablet edb76b7961844443a9b41353c2d77d5a (DEFAULT_TABLE table=default.Table_dda9e1fe74264c2eab25cecc65079c93 [id=50e792c4b9044685ab0605ef0a9474a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.296430 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb76b7961844443a9b41353c2d77d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.298537 1171 ts_tablet_manager.cc:1929] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.298667 1171 log.cc:1199] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/b2f4aff8bf0a43629e43305020df7fa3
I20260505 07:53:53.299003 1171 ts_tablet_manager.cc:1950] T b2f4aff8bf0a43629e43305020df7fa3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:53.299876 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 05b8c0a2185e473cbb4165fed9e94c8a (DEFAULT_TABLE table=default.Table_37f54e4b91df4ca9bcbe16c5b52f6ba8 [id=ad11a4d7ee734c9bab8278aebac5cb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.300194 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b8c0a2185e473cbb4165fed9e94c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.300535 1171 tablet_replica.cc:333] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:53.300747 1171 raft_consensus.cc:2243] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:53.300927 1171 raft_consensus.cc:2272] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:53:53.303736 1103 tablet.cc:2404] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260505 07:53:53.304853 1029 tablet_service.cc:1511] Processing CreateTablet for tablet 8713b273c7b8438d8749f595ee5f1629 (DEFAULT_TABLE table=default.Table_aa53055d0748464697a41feba3c4f036 [id=af8d003d14d24f34bd57e804562d1f33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.305166 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8713b273c7b8438d8749f595ee5f1629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.306702 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a2b859a5dd54bc8b36f3197b58afcea with delete_type TABLET_DATA_DELETED (replaced by tablet 8ca4cf248adb4f768c5586d72614ad42) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.307031 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.307144 1175 ts_tablet_manager.cc:1929] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.307335 1175 log.cc:1199] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/b2f4aff8bf0a43629e43305020df7fa3
I20260505 07:53:53.307874 1175 ts_tablet_manager.cc:1950] T b2f4aff8bf0a43629e43305020df7fa3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:53.308208 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a2b859a5dd54bc8b36f3197b58afcea with delete_type TABLET_DATA_DELETED (replaced by tablet 8ca4cf248adb4f768c5586d72614ad42) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.308662 761 tablet_service.cc:1511] Processing CreateTablet for tablet b7cce14283e7405690a15e26be8a45b3 (DEFAULT_TABLE table=default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.308957 762 tablet_service.cc:1511] Processing CreateTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a (DEFAULT_TABLE table=default.Table_7f23cbfec9794a2bbc9e9dad2d07011c [id=b25c09d8fb8347af83c69e76a523867e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.309130 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cce14283e7405690a15e26be8a45b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.309371 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b151ff45804fbaa76cfbec86c6fd7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.309737 760 tablet_service.cc:1511] Processing CreateTablet for tablet edb76b7961844443a9b41353c2d77d5a (DEFAULT_TABLE table=default.Table_dda9e1fe74264c2eab25cecc65079c93 [id=50e792c4b9044685ab0605ef0a9474a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.310407 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb76b7961844443a9b41353c2d77d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.312166 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 370be7a513ee49028c8d0662b43c8d91 (DEFAULT_TABLE table=default.Table_448efd98249942379556e125b25d169c [id=b2adaf7422c7403c8b331c0d33f0ad53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.312496 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 370be7a513ee49028c8d0662b43c8d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.315934 751 tablet_service.cc:1511] Processing CreateTablet for tablet 370be7a513ee49028c8d0662b43c8d91 (DEFAULT_TABLE table=default.Table_448efd98249942379556e125b25d169c [id=b2adaf7422c7403c8b331c0d33f0ad53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.316304 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 370be7a513ee49028c8d0662b43c8d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.316870 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a2b859a5dd54bc8b36f3197b58afcea with delete_type TABLET_DATA_DELETED (replaced by tablet 8ca4cf248adb4f768c5586d72614ad42) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.317163 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet b2f4aff8bf0a43629e43305020df7fa3 (table default.Table_5e3e2127a0634fc9a6b3fa6d6523d697 [id=d524458d873a48cb980e2f85490232d6]) successfully deleted
W20260505 07:53:53.317406 588 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 0fd990e80a90418dbb9b94cb6e2ee156 because the tablet was not found. No further retry: Not found: Tablet not found: 0fd990e80a90418dbb9b94cb6e2ee156
I20260505 07:53:53.317364 896 tablet_service.cc:1511] Processing CreateTablet for tablet 05b8c0a2185e473cbb4165fed9e94c8a (DEFAULT_TABLE table=default.Table_37f54e4b91df4ca9bcbe16c5b52f6ba8 [id=ad11a4d7ee734c9bab8278aebac5cb40]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:53.317654 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 8087cab2da594607a0c3f7cf5839c6f3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8087cab2da594607a0c3f7cf5839c6f3 already in progress: creating tablet
I20260505 07:53:53.317763 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b8c0a2185e473cbb4165fed9e94c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.318164 1026 tablet_service.cc:1511] Processing CreateTablet for tablet b7cce14283e7405690a15e26be8a45b3 (DEFAULT_TABLE table=default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.318542 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cce14283e7405690a15e26be8a45b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.319975 895 tablet_service.cc:1511] Processing CreateTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a (DEFAULT_TABLE table=default.Table_7f23cbfec9794a2bbc9e9dad2d07011c [id=b25c09d8fb8347af83c69e76a523867e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.320405 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b151ff45804fbaa76cfbec86c6fd7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.322003 897 tablet_service.cc:1511] Processing CreateTablet for tablet 8713b273c7b8438d8749f595ee5f1629 (DEFAULT_TABLE table=default.Table_aa53055d0748464697a41feba3c4f036 [id=af8d003d14d24f34bd57e804562d1f33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.322332 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8713b273c7b8438d8749f595ee5f1629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.323930 901 tablet_service.cc:1511] Processing CreateTablet for tablet b7cce14283e7405690a15e26be8a45b3 (DEFAULT_TABLE table=default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.324358 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cce14283e7405690a15e26be8a45b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.327910 894 tablet_service.cc:1511] Processing CreateTablet for tablet 370be7a513ee49028c8d0662b43c8d91 (DEFAULT_TABLE table=default.Table_448efd98249942379556e125b25d169c [id=b2adaf7422c7403c8b331c0d33f0ad53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.328001 899 tablet_service.cc:1511] Processing CreateTablet for tablet edb76b7961844443a9b41353c2d77d5a (DEFAULT_TABLE table=default.Table_dda9e1fe74264c2eab25cecc65079c93 [id=50e792c4b9044685ab0605ef0a9474a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.328236 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 370be7a513ee49028c8d0662b43c8d91. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:53.328617 830 tablet.cc:2404] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260505 07:53:53.308710 757 tablet_service.cc:1511] Processing CreateTablet for tablet 8713b273c7b8438d8749f595ee5f1629 (DEFAULT_TABLE table=default.Table_aa53055d0748464697a41feba3c4f036 [id=af8d003d14d24f34bd57e804562d1f33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.329798 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8713b273c7b8438d8749f595ee5f1629. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.331874 754 tablet_service.cc:1511] Processing CreateTablet for tablet 05b8c0a2185e473cbb4165fed9e94c8a (DEFAULT_TABLE table=default.Table_37f54e4b91df4ca9bcbe16c5b52f6ba8 [id=ad11a4d7ee734c9bab8278aebac5cb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.332265 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05b8c0a2185e473cbb4165fed9e94c8a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:53.332477 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 6a2b859a5dd54bc8b36f3197b58afcea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6a2b859a5dd54bc8b36f3197b58afcea already in progress: creating tablet
W20260505 07:53:53.332696 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 31b151ff45804fbaa76cfbec86c6fd7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31b151ff45804fbaa76cfbec86c6fd7a already in progress: creating tablet
I20260505 07:53:53.335716 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edb76b7961844443a9b41353c2d77d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.336966 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d443f3293a474454aab0dcda5a03b218"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.337527 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d443f3293a474454aab0dcda5a03b218 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.340137 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_18d810775f644efc92bc9e6f7bada454"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:53.340492 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_8a35dff4878b4429b4a6600b8259fd83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.340641 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_18d810775f644efc92bc9e6f7bada454 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.340833 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ae9bf3e293014ec781c8885d8a716e83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.340996 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a35dff4878b4429b4a6600b8259fd83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:53.341291 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae9bf3e293014ec781c8885d8a716e83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.342289 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_38239693c58a4c8095e38c507eaf96ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.342762 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38239693c58a4c8095e38c507eaf96ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.342880 762 tablet_service.cc:1511] Processing CreateTablet for tablet 772a65f4016843d18fbca25842f76de0 (DEFAULT_TABLE table=default.Table_6208b89234f342409bfac888cc6cfe94 [id=035a76f8e6f34bc1bb1c75c4908bb4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.343254 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772a65f4016843d18fbca25842f76de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.343923 758 tablet_service.cc:1511] Processing CreateTablet for tablet 09a8dbb7fe4b406d804c0b62ba006adb (DEFAULT_TABLE table=default.Table_e78ac004e1af4644a442dee2c23ac5c4 [id=789cc9783535475e9929332f36141e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.344327 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a8dbb7fe4b406d804c0b62ba006adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.347457 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 8ca4cf248adb4f768c5586d72614ad42 (DEFAULT_TABLE table=default.Table_ae1a7c2a1099433289c88a814e0b132e [id=b8e0010041cc477d9ce226cca5a28a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.347908 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca4cf248adb4f768c5586d72614ad42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.347867 759 tablet_service.cc:1511] Processing CreateTablet for tablet 8ca4cf248adb4f768c5586d72614ad42 (DEFAULT_TABLE table=default.Table_ae1a7c2a1099433289c88a814e0b132e [id=b8e0010041cc477d9ce226cca5a28a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.348184 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca4cf248adb4f768c5586d72614ad42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.350477 756 tablet_service.cc:1511] Processing CreateTablet for tablet 2850a301ac4f435a96742b00bf54baa9 (DEFAULT_TABLE table=default.Table_5e3e2127a0634fc9a6b3fa6d6523d697 [id=d524458d873a48cb980e2f85490232d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.350827 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2850a301ac4f435a96742b00bf54baa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.351994 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 09a8dbb7fe4b406d804c0b62ba006adb (DEFAULT_TABLE table=default.Table_e78ac004e1af4644a442dee2c23ac5c4 [id=789cc9783535475e9929332f36141e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.340646 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d65832ff7a8d4b729530e21a683549ab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:53.352380 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a8dbb7fe4b406d804c0b62ba006adb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:53.352629 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d65832ff7a8d4b729530e21a683549ab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.354095 593 master_service.cc:954] ReplaceTablet: received request to replace tablet e1de414720ea44bca392e13c626284bc from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.356092 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 2850a301ac4f435a96742b00bf54baa9 (DEFAULT_TABLE table=default.Table_5e3e2127a0634fc9a6b3fa6d6523d697 [id=d524458d873a48cb980e2f85490232d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.356521 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2850a301ac4f435a96742b00bf54baa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.360986 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 772a65f4016843d18fbca25842f76de0 (DEFAULT_TABLE table=default.Table_6208b89234f342409bfac888cc6cfe94 [id=035a76f8e6f34bc1bb1c75c4908bb4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.361486 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772a65f4016843d18fbca25842f76de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.365649 897 tablet_service.cc:1511] Processing CreateTablet for tablet 2850a301ac4f435a96742b00bf54baa9 (DEFAULT_TABLE table=default.Table_5e3e2127a0634fc9a6b3fa6d6523d697 [id=d524458d873a48cb980e2f85490232d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.366158 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2850a301ac4f435a96742b00bf54baa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.369182 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7cdbf8c75e364774b5e3db4310b3a1e0" } new_table_name: "default.Table_efa325aff89944cc9d011916a21d132e" modify_external_catalogs: true
I20260505 07:53:53.370621 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_26dcd25c1a124d12a99d4c5a02bb4223" } new_table_name: "default.Table_b3f0f5990c0946598233d0bd2fe0084d" modify_external_catalogs: true
I20260505 07:53:53.371918 894 tablet_service.cc:1511] Processing CreateTablet for tablet 8ca4cf248adb4f768c5586d72614ad42 (DEFAULT_TABLE table=default.Table_ae1a7c2a1099433289c88a814e0b132e [id=b8e0010041cc477d9ce226cca5a28a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.372069 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_b6b237d0b9e54608b6c7b015becc676d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:53.372334 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ca4cf248adb4f768c5586d72614ad42. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:53.372628 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b6b237d0b9e54608b6c7b015becc676d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.374655 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_90e7403dad6348e9a3b45c777dfffa98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.375139 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90e7403dad6348e9a3b45c777dfffa98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.378841 896 tablet_service.cc:1511] Processing CreateTablet for tablet 09a8dbb7fe4b406d804c0b62ba006adb (DEFAULT_TABLE table=default.Table_e78ac004e1af4644a442dee2c23ac5c4 [id=789cc9783535475e9929332f36141e27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.379323 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09a8dbb7fe4b406d804c0b62ba006adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.383507 899 tablet_service.cc:1511] Processing CreateTablet for tablet 772a65f4016843d18fbca25842f76de0 (DEFAULT_TABLE table=default.Table_6208b89234f342409bfac888cc6cfe94 [id=035a76f8e6f34bc1bb1c75c4908bb4d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.384092 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 772a65f4016843d18fbca25842f76de0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:53.388533 990 connection.cc:570] client connection to 127.0.105.61:33565 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260505 07:53:53.388945 1102 heartbeater.cc:646] Failed to heartbeat to 127.0.105.61:33565 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260505 07:53:53.398736 971 heartbeater.cc:646] Failed to heartbeat to 127.0.105.61:33565 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.0.105.61:33565 (error 108)
I20260505 07:53:53.406725 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_5e3e2127a0634fc9a6b3fa6d6523d697" } new_table_name: "default.Table_74bdaa65d97949c9ae16b85c62dbcc59" modify_external_catalogs: true
W20260505 07:53:53.408354 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 56a783ccd69d453ea7e7c004bf55170e consensus state has no leader
W20260505 07:53:53.408839 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f296602a34634c649d2573cb178b7053 consensus state has no leader
W20260505 07:53:53.410719 829 heartbeater.cc:646] Failed to heartbeat to 127.0.105.61:33565 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.0.105.61:33565 (error 108)
I20260505 07:53:53.422155 900 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.422832 900 tablet_service.cc:1511] Processing CreateTablet for tablet fea61fad19b34ffaa0b9df3a51979b18 (DEFAULT_TABLE table=default.Table_d443f3293a474454aab0dcda5a03b218 [id=d53c653124364ea5acc357977c9db864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.423292 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea61fad19b34ffaa0b9df3a51979b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.425370 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_da224a28c7e8451a8f4c04fcd2160c6b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.425882 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da224a28c7e8451a8f4c04fcd2160c6b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.427196 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6208b89234f342409bfac888cc6cfe94" } new_table_name: "default.Table_4805d11726a3446b86b7a990c0dbb3c5" modify_external_catalogs: true
I20260505 07:53:53.428455 593 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_37f54e4b91df4ca9bcbe16c5b52f6ba8" } new_table_name: "default.Table_1510db6bb88343d0aa4e95a01ecf17a6" modify_external_catalogs: true
I20260505 07:53:53.431835 758 tablet_service.cc:1511] Processing CreateTablet for tablet fea61fad19b34ffaa0b9df3a51979b18 (DEFAULT_TABLE table=default.Table_d443f3293a474454aab0dcda5a03b218 [id=d53c653124364ea5acc357977c9db864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.432847 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea61fad19b34ffaa0b9df3a51979b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.432989 599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_dda9e1fe74264c2eab25cecc65079c93" } modify_external_catalogs: true
I20260505 07:53:53.433148 599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_dda9e1fe74264c2eab25cecc65079c93" } modify_external_catalogs: true
I20260505 07:53:53.436208 600 master_service.cc:954] ReplaceTablet: received request to replace tablet 05b8c0a2185e473cbb4165fed9e94c8a from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.436817 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 370be7a513ee49028c8d0662b43c8d91 from {username='slave'} at 127.0.0.1:47238
W20260505 07:53:53.436781 587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.105.61:33565 (error 108)
W20260505 07:53:53.439618 1196 negotiation.cc:336] Failed RPC negotiation. Trace:
0505 07:53:53.401649 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.0.105.61:33565 (local address 127.0.105.2:53485)
0505 07:53:53.409537 (+ 7888us) negotiation.cc:107] Waiting for socket to connect
0505 07:53:53.409560 (+ 23us) client_negotiation.cc:175] Beginning negotiation
0505 07:53:53.409692 (+ 132us) client_negotiation.cc:253] Sending NEGOTIATE NegotiatePB request
0505 07:53:53.438514 (+ 28822us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":7745,"thread_start_us":8137,"threads_started":1}
I20260505 07:53:53.443121 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8a35dff4878b4429b4a6600b8259fd83" } modify_external_catalogs: true
I20260505 07:53:53.443326 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8a35dff4878b4429b4a6600b8259fd83" } modify_external_catalogs: true
I20260505 07:53:53.443902 598 master_service.cc:954] ReplaceTablet: received request to replace tablet 689792d9ea734e0bbf1df1c74dab6f83 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.444696 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_0e71e56d0528481eb81fbca5fe427f53"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.445268 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0e71e56d0528481eb81fbca5fe427f53 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:53.448385 1195 negotiation.cc:336] Failed RPC negotiation. Trace:
0505 07:53:53.389836 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.0.105.61:33565 (local address 127.0.105.3:49453)
0505 07:53:53.446925 (+ 57089us) negotiation.cc:107] Waiting for socket to connect
0505 07:53:53.447091 (+ 166us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":56947,"threads_started":1}
I20260505 07:53:53.448340 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2dba7ce60556464488c3d723065cbbfd" } modify_external_catalogs: true
I20260505 07:53:53.448518 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2dba7ce60556464488c3d723065cbbfd" } modify_external_catalogs: true
I20260505 07:53:53.449486 1034 tablet_service.cc:1511] Processing CreateTablet for tablet fea61fad19b34ffaa0b9df3a51979b18 (DEFAULT_TABLE table=default.Table_d443f3293a474454aab0dcda5a03b218 [id=d53c653124364ea5acc357977c9db864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.450023 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fea61fad19b34ffaa0b9df3a51979b18. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:53.451026 1197 negotiation.cc:336] Failed RPC negotiation. Trace:
0505 07:53:53.411243 (+ 0us) reactor.cc:678] Submitting negotiation task for client connection to 127.0.105.61:33565 (local address 127.0.105.1:53369)
0505 07:53:53.450000 (+ 38757us) negotiation.cc:107] Waiting for socket to connect
0505 07:53:53.450176 (+ 176us) negotiation.cc:326] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":38637,"threads_started":1}
I20260505 07:53:53.461135 753 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.462538 1171 ts_tablet_manager.cc:1916] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:53.470093 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2850a301ac4f435a96742b00bf54baa9 consensus state has no leader
I20260505 07:53:53.471685 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ae9bf3e293014ec781c8885d8a716e83" } modify_external_catalogs: true
I20260505 07:53:53.471906 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ae9bf3e293014ec781c8885d8a716e83" } modify_external_catalogs: true
I20260505 07:53:53.485596 896 tablet_service.cc:1511] Processing CreateTablet for tablet 5b124eda66364b7a81426934b98bca48 (DEFAULT_TABLE table=default.Table_d65832ff7a8d4b729530e21a683549ab [id=b4934f65dfeb4a679111783301bb1fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.486066 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b124eda66364b7a81426934b98bca48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.495306 894 tablet_service.cc:1511] Processing CreateTablet for tablet cf95f4331e4540ae84e329cf577427b8 (DEFAULT_TABLE table=default.Table_8a35dff4878b4429b4a6600b8259fd83 [id=0e331c73bbf842dcac6c971a6688f58b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.495538 899 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6490f568024510ab843ec5205a4d1a (DEFAULT_TABLE table=default.Table_ae9bf3e293014ec781c8885d8a716e83 [id=9c204b77f3df4bf9a58508967c89c9f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.495858 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf95f4331e4540ae84e329cf577427b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.496258 898 tablet_service.cc:1511] Processing CreateTablet for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (DEFAULT_TABLE table=default.Table_38239693c58a4c8095e38c507eaf96ec [id=b86b2850fe6f49b1907835ae468cf22e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.496711 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de9e0fe8eb7477ab1cfa64b808575d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.497548 751 tablet_service.cc:1511] Processing CreateTablet for tablet cf95f4331e4540ae84e329cf577427b8 (DEFAULT_TABLE table=default.Table_8a35dff4878b4429b4a6600b8259fd83 [id=0e331c73bbf842dcac6c971a6688f58b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.497938 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf95f4331e4540ae84e329cf577427b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.498836 758 tablet_service.cc:1511] Processing CreateTablet for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (DEFAULT_TABLE table=default.Table_38239693c58a4c8095e38c507eaf96ec [id=b86b2850fe6f49b1907835ae468cf22e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.499208 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de9e0fe8eb7477ab1cfa64b808575d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.499867 759 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec347e754ab4b7282e68b15a9bbf993 (DEFAULT_TABLE table=default.Table_18d810775f644efc92bc9e6f7bada454 [id=7e62bbf111c94a01b324b800bd6ced0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.500326 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec347e754ab4b7282e68b15a9bbf993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.502839 757 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6490f568024510ab843ec5205a4d1a (DEFAULT_TABLE table=default.Table_ae9bf3e293014ec781c8885d8a716e83 [id=9c204b77f3df4bf9a58508967c89c9f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.503194 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6490f568024510ab843ec5205a4d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.504698 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6490f568024510ab843ec5205a4d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.507822 754 tablet_service.cc:1511] Processing CreateTablet for tablet 5b124eda66364b7a81426934b98bca48 (DEFAULT_TABLE table=default.Table_d65832ff7a8d4b729530e21a683549ab [id=b4934f65dfeb4a679111783301bb1fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.508157 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b124eda66364b7a81426934b98bca48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.511986 760 tablet_service.cc:1511] Processing CreateTablet for tablet 62d4d96bdf674c2395938b9971c0ad28 (DEFAULT_TABLE table=default.Table_90e7403dad6348e9a3b45c777dfffa98 [id=c3a480b747de444aa45b771244908352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.512347 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d4d96bdf674c2395938b9971c0ad28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.512897 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet b2f4aff8bf0a43629e43305020df7fa3 (table default.Table_74bdaa65d97949c9ae16b85c62dbcc59 [id=d524458d873a48cb980e2f85490232d6]) successfully deleted
I20260505 07:53:53.515877 753 tablet_service.cc:1511] Processing CreateTablet for tablet 222c41e6cd8b4d0093fd692618efb107 (DEFAULT_TABLE table=default.Table_b6b237d0b9e54608b6c7b015becc676d [id=8f533b7aa2374c439f7234a373af31bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.516201 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 222c41e6cd8b4d0093fd692618efb107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.518379 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 62d4d96bdf674c2395938b9971c0ad28 (DEFAULT_TABLE table=default.Table_90e7403dad6348e9a3b45c777dfffa98 [id=c3a480b747de444aa45b771244908352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.518899 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d4d96bdf674c2395938b9971c0ad28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.519729 895 tablet_service.cc:1511] Processing CreateTablet for tablet 62d4d96bdf674c2395938b9971c0ad28 (DEFAULT_TABLE table=default.Table_90e7403dad6348e9a3b45c777dfffa98 [id=c3a480b747de444aa45b771244908352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.520105 897 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec347e754ab4b7282e68b15a9bbf993 (DEFAULT_TABLE table=default.Table_18d810775f644efc92bc9e6f7bada454 [id=7e62bbf111c94a01b324b800bd6ced0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.520629 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec347e754ab4b7282e68b15a9bbf993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.524551 1176 tablet_bootstrap.cc:492] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:53.524746 1176 ts_tablet_manager.cc:1403] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.378s user 0.076s sys 0.087s
I20260505 07:53:53.525599 1176 raft_consensus.cc:359] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.525802 1176 raft_consensus.cc:385] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.525875 1176 raft_consensus.cc:740] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.526125 1176 consensus_queue.cc:260] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.526623 1176 ts_tablet_manager.cc:1434] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:53:53.526995 1176 tablet_bootstrap.cc:492] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:53.528533 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 222c41e6cd8b4d0093fd692618efb107 (DEFAULT_TABLE table=default.Table_b6b237d0b9e54608b6c7b015becc676d [id=8f533b7aa2374c439f7234a373af31bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.528795 1176 tablet_bootstrap.cc:654] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.529016 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 222c41e6cd8b4d0093fd692618efb107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.533469 901 tablet_service.cc:1511] Processing CreateTablet for tablet 222c41e6cd8b4d0093fd692618efb107 (DEFAULT_TABLE table=default.Table_b6b237d0b9e54608b6c7b015becc676d [id=8f533b7aa2374c439f7234a373af31bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.533926 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 222c41e6cd8b4d0093fd692618efb107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.536150 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 5b124eda66364b7a81426934b98bca48 (DEFAULT_TABLE table=default.Table_d65832ff7a8d4b729530e21a683549ab [id=b4934f65dfeb4a679111783301bb1fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.536584 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b124eda66364b7a81426934b98bca48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.539752 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d4d96bdf674c2395938b9971c0ad28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.541538 1176 tablet_bootstrap.cc:492] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:53.541695 1176 ts_tablet_manager.cc:1403] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.006s
I20260505 07:53:53.542529 1176 raft_consensus.cc:359] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.542773 1176 raft_consensus.cc:385] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.542867 1176 raft_consensus.cc:740] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.543092 1176 consensus_queue.cc:260] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.545869 1176 ts_tablet_manager.cc:1434] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:53:53.548226 1176 tablet_bootstrap.cc:492] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:53.549968 1176 tablet_bootstrap.cc:654] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.553236 1172 tablet_bootstrap.cc:492] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:53.553440 1172 ts_tablet_manager.cc:1403] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.347s user 0.030s sys 0.118s
I20260505 07:53:53.554261 1172 raft_consensus.cc:359] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.554517 1172 raft_consensus.cc:385] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.554644 1172 raft_consensus.cc:740] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.554921 1172 consensus_queue.cc:260] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.556813 1172 ts_tablet_manager.cc:1434] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260505 07:53:53.557299 1172 tablet_bootstrap.cc:492] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:53.559057 1172 tablet_bootstrap.cc:654] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:53.563468 587 consensus_peers.cc:597] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 -> Peer 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565): Couldn't send request to peer 23c57a2c927c4c54bccecfeb3a775311. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260505 07:53:53.563776 1171 ts_tablet_manager.cc:1929] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.563900 1171 log.cc:1199] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/689792d9ea734e0bbf1df1c74dab6f83
I20260505 07:53:53.563993 1026 tablet_service.cc:1511] Processing CreateTablet for tablet cf95f4331e4540ae84e329cf577427b8 (DEFAULT_TABLE table=default.Table_8a35dff4878b4429b4a6600b8259fd83 [id=0e331c73bbf842dcac6c971a6688f58b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.564278 1171 ts_tablet_manager.cc:1950] T 689792d9ea734e0bbf1df1c74dab6f83 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:53.564519 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf95f4331e4540ae84e329cf577427b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.565083 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 4d6490f568024510ab843ec5205a4d1a (DEFAULT_TABLE table=default.Table_ae9bf3e293014ec781c8885d8a716e83 [id=9c204b77f3df4bf9a58508967c89c9f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.565450 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6490f568024510ab843ec5205a4d1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.567971 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 2ec347e754ab4b7282e68b15a9bbf993 (DEFAULT_TABLE table=default.Table_18d810775f644efc92bc9e6f7bada454 [id=7e62bbf111c94a01b324b800bd6ced0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.568388 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ec347e754ab4b7282e68b15a9bbf993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.569006 1171 tablet_replica.cc:333] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:53.569188 1171 raft_consensus.cc:2243] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:53.569330 1171 raft_consensus.cc:2272] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:53.571960 1029 tablet_service.cc:1511] Processing CreateTablet for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (DEFAULT_TABLE table=default.Table_38239693c58a4c8095e38c507eaf96ec [id=b86b2850fe6f49b1907835ae468cf22e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.572978 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 689792d9ea734e0bbf1df1c74dab6f83 (table default.Table_8b2e2e36529648cebd2bbdeda19b23b6 [id=0407ce7e0583423e92f51c513897877d]) successfully deleted
W20260505 07:53:53.573366 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a25715f0fa504e92b54c48712c2c2f48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a25715f0fa504e92b54c48712c2c2f48 already in progress: creating tablet
I20260505 07:53:53.585515 1171 ts_tablet_manager.cc:1916] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:53.590167 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 8087cab2da594607a0c3f7cf5839c6f3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8087cab2da594607a0c3f7cf5839c6f3 already in progress: creating tablet
W20260505 07:53:53.591001 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet a25715f0fa504e92b54c48712c2c2f48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a25715f0fa504e92b54c48712c2c2f48 already in progress: creating tablet
W20260505 07:53:53.592072 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 31b151ff45804fbaa76cfbec86c6fd7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31b151ff45804fbaa76cfbec86c6fd7a already in progress: creating tablet
I20260505 07:53:53.592620 1172 tablet_bootstrap.cc:492] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:53.592872 1172 ts_tablet_manager.cc:1403] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.004s
I20260505 07:53:53.593932 1175 tablet_replica.cc:333] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:53.593842 1172 raft_consensus.cc:359] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.594142 1172 raft_consensus.cc:385] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.594152 1175 raft_consensus.cc:2243] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:53.594353 1175 raft_consensus.cc:2272] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:53.595283 1175 ts_tablet_manager.cc:1916] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:53.597815 1176 tablet_bootstrap.cc:492] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:53.598038 1176 ts_tablet_manager.cc:1403] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.050s user 0.005s sys 0.005s
I20260505 07:53:53.598894 1176 raft_consensus.cc:359] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.599152 1176 raft_consensus.cc:385] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.599284 1176 raft_consensus.cc:740] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.599632 1176 consensus_queue.cc:260] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.600116 751 tablet_service.cc:1558] Processing DeleteTablet for tablet e1de414720ea44bca392e13c626284bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.602241 1176 ts_tablet_manager.cc:1434] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260505 07:53:53.605718 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de9e0fe8eb7477ab1cfa64b808575d8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:53.607756 593 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 05b8c0a2185e473cbb4165fed9e94c8a consensus state has no leader
W20260505 07:53:53.607944 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 772a65f4016843d18fbca25842f76de0 consensus state has no leader
I20260505 07:53:53.608359 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b3f0f5990c0946598233d0bd2fe0084d" } modify_external_catalogs: true
I20260505 07:53:53.608613 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b3f0f5990c0946598233d0bd2fe0084d" } modify_external_catalogs: true
I20260505 07:53:53.608901 593 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 56a783ccd69d453ea7e7c004bf55170e (table default.Table_efa325aff89944cc9d011916a21d132e [id=dda64712b2cd4f1286abaf5c08022180]). Expected version 3 got 0
I20260505 07:53:53.609725 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 5b124eda66364b7a81426934b98bca48 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.614431 1179 raft_consensus.cc:493] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:53.614681 1179 raft_consensus.cc:515] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.616950 1179 leader_election.cc:290] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:53.630926 1181 tablet_bootstrap.cc:492] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:53.631086 1181 ts_tablet_manager.cc:1403] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.390s user 0.054s sys 0.109s
I20260505 07:53:53.632287 1181 raft_consensus.cc:359] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.632498 1181 raft_consensus.cc:385] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.632581 1181 raft_consensus.cc:740] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.632900 1181 consensus_queue.cc:260] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.633477 1181 ts_tablet_manager.cc:1434] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260505 07:53:53.633740 1181 tablet_bootstrap.cc:492] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:53.635752 1181 tablet_bootstrap.cc:654] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.648106 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 2850a301ac4f435a96742b00bf54baa9 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.653450 1179 raft_consensus.cc:493] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:53.653688 1179 raft_consensus.cc:515] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.655265 1179 leader_election.cc:290] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:53.666596 1176 tablet_bootstrap.cc:492] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:53.669591 1176 tablet_bootstrap.cc:654] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.594393 1172 raft_consensus.cc:740] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.672992 1172 consensus_queue.cc:260] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.676442 1172 ts_tablet_manager.cc:1434] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.083s user 0.001s sys 0.001s
I20260505 07:53:53.679906 1172 tablet_bootstrap.cc:492] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:53.682772 1180 raft_consensus.cc:493] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:53.683056 1180 raft_consensus.cc:515] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.685665 1180 leader_election.cc:290] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:53.724664 1191 raft_consensus.cc:493] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:53.724957 1191 raft_consensus.cc:515] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.727273 1191 leader_election.cc:290] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:53.732854 1179 raft_consensus.cc:493] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:53.733085 1179 raft_consensus.cc:515] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.733912 1179 leader_election.cc:290] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:53.753182 1172 tablet_bootstrap.cc:654] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:53.768257 588 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 56a783ccd69d453ea7e7c004bf55170e (table default.Table_efa325aff89944cc9d011916a21d132e [id=dda64712b2cd4f1286abaf5c08022180], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 56a783ccd69d453ea7e7c004bf55170e consensus state has no leader
I20260505 07:53:53.770128 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet edb76b7961844443a9b41353c2d77d5a
I20260505 07:53:53.770995 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet edb76b7961844443a9b41353c2d77d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.771014 895 tablet_service.cc:1558] Processing DeleteTablet for tablet edb76b7961844443a9b41353c2d77d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:53.771674 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet edb76b7961844443a9b41353c2d77d5a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edb76b7961844443a9b41353c2d77d5a already in progress: creating tablet
I20260505 07:53:53.771896 1171 ts_tablet_manager.cc:1929] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.772107 1171 log.cc:1199] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/070d1b1b30d447539911349841638c12
I20260505 07:53:53.772548 1171 ts_tablet_manager.cc:1950] T 070d1b1b30d447539911349841638c12 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:53.772746 600 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 05b8c0a2185e473cbb4165fed9e94c8a
I20260505 07:53:53.773268 600 catalog_manager.cc:6741] ReplaceTablet: tablet 05b8c0a2185e473cbb4165fed9e94c8a deleted and replaced by tablet 5f57568e1c4c4573ac18d9f621eeb4c7
I20260505 07:53:53.773610 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 05b8c0a2185e473cbb4165fed9e94c8a with delete_type TABLET_DATA_DELETED (replaced by tablet 5f57568e1c4c4573ac18d9f621eeb4c7) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.774186 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 05b8c0a2185e473cbb4165fed9e94c8a with delete_type TABLET_DATA_DELETED (replaced by tablet 5f57568e1c4c4573ac18d9f621eeb4c7) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.774253 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 070d1b1b30d447539911349841638c12 (table default.Table_8c90ac7761034713834e4430232a973e [id=9fcb3767fb974e7fba21836bf2a06dc6]) successfully deleted
I20260505 07:53:53.774293 1171 tablet_replica.cc:333] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
W20260505 07:53:53.775347 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 05b8c0a2185e473cbb4165fed9e94c8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05b8c0a2185e473cbb4165fed9e94c8a already in progress: creating tablet
I20260505 07:53:53.776438 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_2b834837c1d54b429b8d9a937f56d726"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.777173 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b834837c1d54b429b8d9a937f56d726 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.779114 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet e1de414720ea44bca392e13c626284bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.779404 1178 tablet_replica.cc:333] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:53.779960 1178 raft_consensus.cc:2243] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:53.780390 1178 raft_consensus.cc:2272] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:53.780646 599 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_e78ac004e1af4644a442dee2c23ac5c4" } new_table_name: "default.Table_eae91d411d9149658200d44f4837f486" modify_external_catalogs: true
I20260505 07:53:53.782413 1178 ts_tablet_manager.cc:1916] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:53.782307 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1de414720ea44bca392e13c626284bc" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:53:53.782718 921 raft_consensus.cc:2468] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:53:53.783656 991 leader_election.cc:304] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
W20260505 07:53:53.784045 991 raft_consensus.cc:2686] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260505 07:53:53.785032 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 370be7a513ee49028c8d0662b43c8d91
I20260505 07:53:53.785576 595 catalog_manager.cc:6741] ReplaceTablet: tablet 370be7a513ee49028c8d0662b43c8d91 deleted and replaced by tablet 5dab02a546b944f799045723a515118f
I20260505 07:53:53.786849 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 370be7a513ee49028c8d0662b43c8d91 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dab02a546b944f799045723a515118f) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.786803 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 370be7a513ee49028c8d0662b43c8d91 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dab02a546b944f799045723a515118f) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.790043 1178 ts_tablet_manager.cc:1929] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.790256 1178 log.cc:1199] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/e1de414720ea44bca392e13c626284bc
I20260505 07:53:53.790378 1179 raft_consensus.cc:493] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:53.790578 1179 raft_consensus.cc:515] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.790889 1178 ts_tablet_manager.cc:1950] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:53.791240 1179 leader_election.cc:290] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:53.791697 759 tablet_service.cc:1558] Processing DeleteTablet for tablet edb76b7961844443a9b41353c2d77d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.791970 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet e1de414720ea44bca392e13c626284bc (table default.Table_1cae231a7d5647bf887cd8dd8bb0ded9 [id=7ba2bfd0de87417c944ea8a4edf99b14]) successfully deleted
I20260505 07:53:53.792088 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 05b8c0a2185e473cbb4165fed9e94c8a with delete_type TABLET_DATA_DELETED (replaced by tablet 5f57568e1c4c4573ac18d9f621eeb4c7) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.792470 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 689792d9ea734e0bbf1df1c74dab6f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:53.792726 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 370be7a513ee49028c8d0662b43c8d91 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 370be7a513ee49028c8d0662b43c8d91 already in progress: creating tablet
I20260505 07:53:53.792865 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 370be7a513ee49028c8d0662b43c8d91 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dab02a546b944f799045723a515118f) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.794384 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f296602a34634c649d2573cb178b7053" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:53:53.795075 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a783ccd69d453ea7e7c004bf55170e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:53:53.795718 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2b859a5dd54bc8b36f3197b58afcea" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:53:53.796195 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet e1de414720ea44bca392e13c626284bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:53.796716 587 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e1de414720ea44bca392e13c626284bc because the tablet was not found. No further retry: Not found: Tablet not found: e1de414720ea44bca392e13c626284bc
I20260505 07:53:53.796973 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_18d810775f644efc92bc9e6f7bada454" } new_table_name: "default.Table_d99fe22bedab4ff5b8635dc36e2f5060" modify_external_catalogs: true
I20260505 07:53:53.798228 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet cf95f4331e4540ae84e329cf577427b8
I20260505 07:53:53.799248 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet cf95f4331e4540ae84e329cf577427b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.799315 1176 tablet_bootstrap.cc:492] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:53.799466 1176 ts_tablet_manager.cc:1403] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.133s user 0.004s sys 0.001s
I20260505 07:53:53.799467 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_468330cd933549a587255653023d8e49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.799817 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet cf95f4331e4540ae84e329cf577427b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf95f4331e4540ae84e329cf577427b8 already in progress: creating tablet
W20260505 07:53:53.800086 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_468330cd933549a587255653023d8e49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.800428 1176 raft_consensus.cc:359] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.800661 1176 raft_consensus.cc:385] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.800763 1176 raft_consensus.cc:740] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.801110 1176 consensus_queue.cc:260] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.801766 895 tablet_service.cc:1558] Processing DeleteTablet for tablet cf95f4331e4540ae84e329cf577427b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.806007 1176 ts_tablet_manager.cc:1434] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260505 07:53:53.808688 1176 tablet_bootstrap.cc:492] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:53.811057 1176 tablet_bootstrap.cc:654] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.815614 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f296602a34634c649d2573cb178b7053" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:53.815693 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a783ccd69d453ea7e7c004bf55170e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:53.816102 1053 raft_consensus.cc:2468] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:53:53.816452 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2b859a5dd54bc8b36f3197b58afcea" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:53.816856 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "689792d9ea734e0bbf1df1c74dab6f83" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:53.818318 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet c70dedbd58ca402daf0fdde68cc5b574
I20260505 07:53:53.819941 895 tablet_service.cc:1558] Processing DeleteTablet for tablet c70dedbd58ca402daf0fdde68cc5b574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.820088 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet c70dedbd58ca402daf0fdde68cc5b574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:53.820746 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet c70dedbd58ca402daf0fdde68cc5b574 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c70dedbd58ca402daf0fdde68cc5b574 already in progress: creating tablet
W20260505 07:53:53.821905 856 leader_election.cc:343] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: e1de414720ea44bca392e13c626284bc
W20260505 07:53:53.822496 717 leader_election.cc:343] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:53.822839 717 leader_election.cc:304] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:53:53.823325 1179 raft_consensus.cc:2804] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:53.823480 1179 raft_consensus.cc:493] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:53.823626 1179 raft_consensus.cc:3060] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:53.824385 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6522a65a3ea840658d0e343d76a64379"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.825067 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6522a65a3ea840658d0e343d76a64379 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.827665 1179 raft_consensus.cc:515] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.828562 1179 leader_election.cc:290] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:53:53.828981 717 leader_election.cc:343] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:53:53.829208 717 leader_election.cc:343] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:53.829773 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a783ccd69d453ea7e7c004bf55170e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:53:53.829981 1055 raft_consensus.cc:3060] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:53.830421 895 tablet_service.cc:1558] Processing DeleteTablet for tablet e1de414720ea44bca392e13c626284bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.831900 1175 ts_tablet_manager.cc:1929] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.832038 1175 log.cc:1199] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/b705bde46048477b92436eea59cde986
I20260505 07:53:53.832376 1175 ts_tablet_manager.cc:1950] T b705bde46048477b92436eea59cde986 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:53.832573 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7d1be5262b41414f8a0fc37fba1cc28a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:53.833084 1055 raft_consensus.cc:2468] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:53:53.833210 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d1be5262b41414f8a0fc37fba1cc28a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.833280 1175 tablet_replica.cc:333] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:53.833647 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet b705bde46048477b92436eea59cde986 (table default.Table_cd6f1e7c73784523be3f560bd458d7b2 [id=caadebc30e5a4c39877cd29df1e35306]) successfully deleted
I20260505 07:53:53.834139 759 tablet_service.cc:1558] Processing DeleteTablet for tablet cf95f4331e4540ae84e329cf577427b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.834436 717 leader_election.cc:304] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:53:53.834583 759 tablet_service.cc:1558] Processing DeleteTablet for tablet c70dedbd58ca402daf0fdde68cc5b574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.834800 1179 raft_consensus.cc:2804] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
W20260505 07:53:53.835289 718 leader_election.cc:343] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 689792d9ea734e0bbf1df1c74dab6f83
I20260505 07:53:53.835435 718 leader_election.cc:304] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:53.836931 1172 tablet_bootstrap.cc:492] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:53.837112 1172 ts_tablet_manager.cc:1403] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.157s user 0.001s sys 0.003s
I20260505 07:53:53.838181 1172 raft_consensus.cc:359] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.838436 1172 raft_consensus.cc:385] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.838552 1172 raft_consensus.cc:740] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.838837 1172 consensus_queue.cc:260] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.839509 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a783ccd69d453ea7e7c004bf55170e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:53:53.841946 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 4d6490f568024510ab843ec5205a4d1a
I20260505 07:53:53.843395 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6490f568024510ab843ec5205a4d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.843477 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6490f568024510ab843ec5205a4d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.843806 1171 raft_consensus.cc:2243] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260505 07:53:53.844391 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 4d6490f568024510ab843ec5205a4d1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d6490f568024510ab843ec5205a4d1a already in progress: creating tablet
I20260505 07:53:53.846192 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_395f836d4acc48deb712909ac7ca320d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.846854 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_395f836d4acc48deb712909ac7ca320d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.855182 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet f296602a34634c649d2573cb178b7053
I20260505 07:53:53.856675 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet f296602a34634c649d2573cb178b7053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.856657 895 tablet_service.cc:1558] Processing DeleteTablet for tablet f296602a34634c649d2573cb178b7053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.857038 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1de414720ea44bca392e13c626284bc" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:53.857530 782 raft_consensus.cc:2468] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
W20260505 07:53:53.857620 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f296602a34634c649d2573cb178b7053 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f296602a34634c649d2573cb178b7053 already in progress: creating tablet
W20260505 07:53:53.858490 718 leader_election.cc:343] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:53.858660 718 leader_election.cc:304] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:53.858973 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_59efa091aa214f2698ba8a102874b9cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:53.859308 854 leader_election.cc:304] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
W20260505 07:53:53.859649 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59efa091aa214f2698ba8a102874b9cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.865938 1171 raft_consensus.cc:2272] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:53.866743 1171 ts_tablet_manager.cc:1916] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:53.866760 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 5b124eda66364b7a81426934b98bca48
I20260505 07:53:53.867266 593 catalog_manager.cc:6741] ReplaceTablet: tablet 5b124eda66364b7a81426934b98bca48 deleted and replaced by tablet 3781d0bbcd1b4189bd836eeae4236f40
I20260505 07:53:53.868265 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b124eda66364b7a81426934b98bca48 with delete_type TABLET_DATA_DELETED (replaced by tablet 3781d0bbcd1b4189bd836eeae4236f40) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.868583 1180 raft_consensus.cc:2804] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:53.868784 1180 raft_consensus.cc:493] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:53.868901 1180 raft_consensus.cc:3060] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:53.869298 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b124eda66364b7a81426934b98bca48 with delete_type TABLET_DATA_DELETED (replaced by tablet 3781d0bbcd1b4189bd836eeae4236f40) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:53.870415 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 5b124eda66364b7a81426934b98bca48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5b124eda66364b7a81426934b98bca48 already in progress: creating tablet
I20260505 07:53:53.872994 1180 raft_consensus.cc:515] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.873093 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_bc8ab89c006e4ff2a0219007794efa64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.873770 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bc8ab89c006e4ff2a0219007794efa64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.873852 1180 leader_election.cc:290] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:53:53.874903 856 leader_election.cc:343] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: e1de414720ea44bca392e13c626284bc
I20260505 07:53:53.875911 1172 ts_tablet_manager.cc:1434] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.039s user 0.001s sys 0.002s
I20260505 07:53:53.877808 1181 tablet_bootstrap.cc:492] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:53.878585 1172 tablet_bootstrap.cc:492] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:53.878021 1181 ts_tablet_manager.cc:1403] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.244s user 0.098s sys 0.104s
I20260505 07:53:53.882210 1172 tablet_bootstrap.cc:654] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:53.883433 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 2850a301ac4f435a96742b00bf54baa9
I20260505 07:53:53.883605 1181 raft_consensus.cc:359] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.883869 1181 raft_consensus.cc:385] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.883960 1181 raft_consensus.cc:740] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.886454 596 catalog_manager.cc:6741] ReplaceTablet: tablet 2850a301ac4f435a96742b00bf54baa9 deleted and replaced by tablet 956b5e05bda648349ba26d5c1ea3b279
I20260505 07:53:53.889053 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_476efdff9cee417b941d0796953265f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.889642 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_476efdff9cee417b941d0796953265f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.889819 1181 consensus_queue.cc:260] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:53:53.890816 599 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09a8dbb7fe4b406d804c0b62ba006adb consensus state has no leader
I20260505 07:53:53.891037 599 master_service.cc:954] ReplaceTablet: received request to replace tablet a25715f0fa504e92b54c48712c2c2f48 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:53.892614 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 2850a301ac4f435a96742b00bf54baa9 with delete_type TABLET_DATA_DELETED (replaced by tablet 956b5e05bda648349ba26d5c1ea3b279) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.893918 1181 ts_tablet_manager.cc:1434] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.014s user 0.000s sys 0.002s
I20260505 07:53:53.895031 1181 tablet_bootstrap.cc:492] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:53.897375 1181 tablet_bootstrap.cc:654] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:53.897658 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ec347e754ab4b7282e68b15a9bbf993 consensus state has no leader
I20260505 07:53:53.898380 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d65832ff7a8d4b729530e21a683549ab" } new_table_name: "default.Table_14478c9067fb42d6a508110c0aa360ed" modify_external_catalogs: true
W20260505 07:53:53.900231 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 2850a301ac4f435a96742b00bf54baa9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2850a301ac4f435a96742b00bf54baa9 already in progress: creating tablet
I20260505 07:53:53.900734 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet b705bde46048477b92436eea59cde986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.901006 1178 tablet_replica.cc:333] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:53.901228 1178 raft_consensus.cc:2243] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:53.901422 1178 raft_consensus.cc:2272] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:53.902173 1178 ts_tablet_manager.cc:1916] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:53.903702 599 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ae1a7c2a1099433289c88a814e0b132e" } new_table_name: "default.Table_31dd1382f8b4494e9e13ac5b3ac18aa0" modify_external_catalogs: true
I20260505 07:53:53.909427 1165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:53.909629 1165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: This master's current role is: LEADER
I20260505 07:53:53.911692 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6e9d636b8b314e5b888c725d34775180"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.912249 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6e9d636b8b314e5b888c725d34775180 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.913363 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d443f3293a474454aab0dcda5a03b218" } new_table_name: "default.Table_d0fa020605d347b3ad7096aed39c552c" modify_external_catalogs: true
I20260505 07:53:53.913769 1178 ts_tablet_manager.cc:1929] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.913930 1178 log.cc:1199] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/b705bde46048477b92436eea59cde986
I20260505 07:53:53.914354 1178 ts_tablet_manager.cc:1950] T b705bde46048477b92436eea59cde986 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:53.916339 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet b705bde46048477b92436eea59cde986 (table default.Table_cd6f1e7c73784523be3f560bd458d7b2 [id=caadebc30e5a4c39877cd29df1e35306]) successfully deleted
I20260505 07:53:53.921396 1181 tablet_bootstrap.cc:492] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:53.921564 1181 ts_tablet_manager.cc:1403] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.027s user 0.001s sys 0.009s
I20260505 07:53:53.921521 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d0573a3888634008ba3c77a6fc99a718"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:53.922140 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0573a3888634008ba3c77a6fc99a718 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:53.922578 1181 raft_consensus.cc:359] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:53.922811 1181 raft_consensus.cc:385] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:53.922905 1181 raft_consensus.cc:740] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:53.923158 1181 consensus_queue.cc:260] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:53.927085 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet b705bde46048477b92436eea59cde986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:53.927830 587 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet b705bde46048477b92436eea59cde986 because the tablet was not found. No further retry: Not found: Tablet not found: b705bde46048477b92436eea59cde986
I20260505 07:53:53.928097 1181 ts_tablet_manager.cc:1434] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260505 07:53:53.929594 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 070d1b1b30d447539911349841638c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:53.930873 1181 tablet_bootstrap.cc:492] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:53:53.932938 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3781d0bbcd1b4189bd836eeae4236f40 has no consensus state
I20260505 07:53:53.933082 1181 tablet_bootstrap.cc:654] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:53.935738 599 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8ca4cf248adb4f768c5586d72614ad42 consensus state has no leader
I20260505 07:53:53.937343 1178 tablet_replica.cc:333] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:53.937696 1178 raft_consensus.cc:2243] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:53.937978 1178 raft_consensus.cc:2272] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:53.938932 1178 ts_tablet_manager.cc:1916] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:53.946038 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1de414720ea44bca392e13c626284bc" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:53:53.946324 782 raft_consensus.cc:3060] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:53.902751 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 2850a301ac4f435a96742b00bf54baa9 with delete_type TABLET_DATA_DELETED (replaced by tablet 956b5e05bda648349ba26d5c1ea3b279) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:53.949786 782 raft_consensus.cc:2468] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
W20260505 07:53:53.949882 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet fea61fad19b34ffaa0b9df3a51979b18 consensus state has no leader
I20260505 07:53:53.950312 854 leader_election.cc:304] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:53.950903 1180 raft_consensus.cc:2804] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:53.951567 1219 raft_consensus.cc:2749] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:53.952059 1220 raft_consensus.cc:2749] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:53.953541 1171 ts_tablet_manager.cc:1929] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.953681 1171 log.cc:1199] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/b705bde46048477b92436eea59cde986
I20260505 07:53:53.955150 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 (DEFAULT_TABLE table=default.Table_2b834837c1d54b429b8d9a937f56d726 [id=49a5c863608b402a9ca94c01e5a5c09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.955631 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.958500 1175 raft_consensus.cc:2243] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:53.958971 1175 raft_consensus.cc:2272] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:53.959988 1175 ts_tablet_manager.cc:1916] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:53.963213 895 tablet_service.cc:1511] Processing CreateTablet for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 (DEFAULT_TABLE table=default.Table_2b834837c1d54b429b8d9a937f56d726 [id=49a5c863608b402a9ca94c01e5a5c09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.963752 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.963733 1179 raft_consensus.cc:697] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:53:53.964303 1179 consensus_queue.cc:237] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.970557 1180 raft_consensus.cc:697] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:53:53.971148 1180 consensus_queue.cc:237] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.972720 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d0573a3888634008ba3c77a6fc99a718" } modify_external_catalogs: true
I20260505 07:53:53.972913 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d0573a3888634008ba3c77a6fc99a718" } modify_external_catalogs: true
I20260505 07:53:53.977607 1027 tablet_service.cc:1511] Processing CreateTablet for tablet ce9d01b0b5464a65966f26839b48aaf1 (DEFAULT_TABLE table=default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.978017 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce9d01b0b5464a65966f26839b48aaf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.978062 1229 raft_consensus.cc:493] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:53.978184 1229 raft_consensus.cc:515] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:53.978953 901 tablet_service.cc:1511] Processing CreateTablet for tablet ce9d01b0b5464a65966f26839b48aaf1 (DEFAULT_TABLE table=default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:53.979323 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70dedbd58ca402daf0fdde68cc5b574" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:53.979390 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce9d01b0b5464a65966f26839b48aaf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:53.979532 782 raft_consensus.cc:2468] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:53:53.979997 854 leader_election.cc:304] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:53:53.980566 1229 leader_election.cc:290] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:53.980762 1229 raft_consensus.cc:2804] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:53.980883 1229 raft_consensus.cc:493] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:53.980979 1229 raft_consensus.cc:3060] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:53.980979 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70dedbd58ca402daf0fdde68cc5b574" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:53.983060 856 leader_election.cc:343] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:53.983917 1229 raft_consensus.cc:515] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:53:53.984740 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 126254us
W20260505 07:53:53.985105 718 leader_election.cc:343] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:53:53.985544 718 leader_election.cc:343] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:53.985733 718 leader_election.cc:304] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
W20260505 07:53:53.986085 718 raft_consensus.cc:2686] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260505 07:53:53.986492 718 leader_election.cc:343] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:53.994238 594 catalog_manager.cc:5671] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:53:53.995138 594 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 56a783ccd69d453ea7e7c004bf55170e (table default.Table_efa325aff89944cc9d011916a21d132e [id=dda64712b2cd4f1286abaf5c08022180]). Expected version 3 got 0
I20260505 07:53:53.995998 1178 ts_tablet_manager.cc:1929] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.996116 1178 log.cc:1199] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/070d1b1b30d447539911349841638c12
I20260505 07:53:53.996627 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6490f568024510ab843ec5205a4d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.997334 759 tablet_service.cc:1558] Processing DeleteTablet for tablet f296602a34634c649d2573cb178b7053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:53.999724 1175 ts_tablet_manager.cc:1929] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:53.999857 1175 log.cc:1199] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/6a2b859a5dd54bc8b36f3197b58afcea
I20260505 07:53:54.000279 1175 ts_tablet_manager.cc:1950] T 6a2b859a5dd54bc8b36f3197b58afcea P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:54.001518 1175 tablet_replica.cc:333] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:54.001797 1175 raft_consensus.cc:2243] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.001979 1175 raft_consensus.cc:2272] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.002293 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b124eda66364b7a81426934b98bca48 with delete_type TABLET_DATA_DELETED (replaced by tablet 3781d0bbcd1b4189bd836eeae4236f40) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.002827 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 2850a301ac4f435a96742b00bf54baa9 with delete_type TABLET_DATA_DELETED (replaced by tablet 956b5e05bda648349ba26d5c1ea3b279) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.002836 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_efa325aff89944cc9d011916a21d132e" } modify_external_catalogs: true
I20260505 07:53:54.002985 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_efa325aff89944cc9d011916a21d132e" } modify_external_catalogs: true
I20260505 07:53:54.003712 759 tablet_service.cc:1511] Processing CreateTablet for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 (DEFAULT_TABLE table=default.Table_2b834837c1d54b429b8d9a937f56d726 [id=49a5c863608b402a9ca94c01e5a5c09f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.004235 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.007097 592 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 5a67fe4d455e4f1b9f195d08d62be8a7
I20260505 07:53:54.009254 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 3781d0bbcd1b4189bd836eeae4236f40 (DEFAULT_TABLE table=default.Table_14478c9067fb42d6a508110c0aa360ed [id=b4934f65dfeb4a679111783301bb1fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.009689 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3781d0bbcd1b4189bd836eeae4236f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.011890 1178 ts_tablet_manager.cc:1950] T 070d1b1b30d447539911349841638c12 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:54.012820 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 070d1b1b30d447539911349841638c12 (table default.Table_8c90ac7761034713834e4430232a973e [id=9fcb3767fb974e7fba21836bf2a06dc6]) successfully deleted
I20260505 07:53:54.013263 759 tablet_service.cc:1511] Processing CreateTablet for tablet ce9d01b0b5464a65966f26839b48aaf1 (DEFAULT_TABLE table=default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.013588 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce9d01b0b5464a65966f26839b48aaf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.013628 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 56cbbb89f6b84407b2c344ec62402b03 (DEFAULT_TABLE table=default.Table_da224a28c7e8451a8f4c04fcd2160c6b [id=bd575286f4e94df3b335b5e79d8f971c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.014061 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56cbbb89f6b84407b2c344ec62402b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.019322 751 tablet_service.cc:1511] Processing CreateTablet for tablet 3781d0bbcd1b4189bd836eeae4236f40 (DEFAULT_TABLE table=default.Table_14478c9067fb42d6a508110c0aa360ed [id=b4934f65dfeb4a679111783301bb1fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.019440 1176 tablet_bootstrap.cc:492] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:54.019605 1176 ts_tablet_manager.cc:1403] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.212s user 0.005s sys 0.000s
I20260505 07:53:54.019754 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3781d0bbcd1b4189bd836eeae4236f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.020609 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 56a783ccd69d453ea7e7c004bf55170e
I20260505 07:53:54.020388 1176 raft_consensus.cc:359] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.020746 1176 raft_consensus.cc:385] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.020843 1176 raft_consensus.cc:740] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.021750 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 56a783ccd69d453ea7e7c004bf55170e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.022104 1176 consensus_queue.cc:260] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.022316 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_bc8ab89c006e4ff2a0219007794efa64" } modify_external_catalogs: true
I20260505 07:53:54.022516 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_bc8ab89c006e4ff2a0219007794efa64" } modify_external_catalogs: true
I20260505 07:53:54.022599 1176 ts_tablet_manager.cc:1434] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:53:54.022940 1176 tablet_bootstrap.cc:492] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:54.024637 1176 tablet_bootstrap.cc:654] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.031756 1176 tablet_bootstrap.cc:492] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:54.031893 1176 ts_tablet_manager.cc:1403] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260505 07:53:54.032713 1176 raft_consensus.cc:359] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:54.032920 1176 raft_consensus.cc:385] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.033021 1176 raft_consensus.cc:740] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.033224 1176 consensus_queue.cc:260] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:54.036782 751 tablet_service.cc:1511] Processing CreateTablet for tablet 56cbbb89f6b84407b2c344ec62402b03 (DEFAULT_TABLE table=default.Table_da224a28c7e8451a8f4c04fcd2160c6b [id=bd575286f4e94df3b335b5e79d8f971c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.037751 1176 ts_tablet_manager.cc:1434] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:53:54.038143 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56cbbb89f6b84407b2c344ec62402b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.039924 1176 tablet_bootstrap.cc:492] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:54.045143 1176 tablet_bootstrap.cc:654] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.045574 594 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 703fc8ad1a31432e99939f5221b40b05
I20260505 07:53:54.046582 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_395f836d4acc48deb712909ac7ca320d" } modify_external_catalogs: true
I20260505 07:53:54.046793 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_395f836d4acc48deb712909ac7ca320d" } modify_external_catalogs: true
I20260505 07:53:54.054409 594 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 764e92dff9244a0fb1deb61265604400
I20260505 07:53:54.055979 1029 tablet_service.cc:1511] Processing CreateTablet for tablet 1765a95cc3414c73b7e3918913a35937 (DEFAULT_TABLE table=default.Table_468330cd933549a587255653023d8e49 [id=13ca7143dd0249d597c79bbfc17ae7f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.056411 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1765a95cc3414c73b7e3918913a35937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.058560 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2b834837c1d54b429b8d9a937f56d726" } modify_external_catalogs: true
I20260505 07:53:54.058753 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2b834837c1d54b429b8d9a937f56d726" } modify_external_catalogs: true
I20260505 07:53:54.061740 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 222c41e6cd8b4d0093fd692618efb107 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:54.062620 1176 tablet_bootstrap.cc:492] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:54.063803 1176 ts_tablet_manager.cc:1403] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.001s
I20260505 07:53:54.064040 753 tablet_service.cc:1511] Processing CreateTablet for tablet 5f57568e1c4c4573ac18d9f621eeb4c7 (DEFAULT_TABLE table=default.Table_1510db6bb88343d0aa4e95a01ecf17a6 [id=ad11a4d7ee734c9bab8278aebac5cb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.064452 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f57568e1c4c4573ac18d9f621eeb4c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.064857 1176 raft_consensus.cc:359] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.065126 1176 raft_consensus.cc:385] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.065248 1176 raft_consensus.cc:740] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.065279 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 8cfe212cb6c14ce3a63b6ee4275f66f1
I20260505 07:53:54.066215 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 5dab02a546b944f799045723a515118f (DEFAULT_TABLE table=default.Table_448efd98249942379556e125b25d169c [id=b2adaf7422c7403c8b331c0d33f0ad53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.066630 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dab02a546b944f799045723a515118f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.067116 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.067652 1176 consensus_queue.cc:260] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.071448 1176 ts_tablet_manager.cc:1434] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260505 07:53:54.072983 1176 tablet_bootstrap.cc:492] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:54.075484 1176 tablet_bootstrap.cc:654] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.075526 753 tablet_service.cc:1558] Processing DeleteTablet for tablet 56a783ccd69d453ea7e7c004bf55170e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.075804 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd990e80a90418dbb9b94cb6e2ee156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.076470 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 2782cf753bc44bf4a102dd785ea3c6b8 (DEFAULT_TABLE table=default.Table_6522a65a3ea840658d0e343d76a64379 [id=37397b87cfe445f2adfb7eb0af27454b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.076833 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2782cf753bc44bf4a102dd785ea3c6b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.077221 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 956b5e05bda648349ba26d5c1ea3b279 (DEFAULT_TABLE table=default.Table_74bdaa65d97949c9ae16b85c62dbcc59 [id=d524458d873a48cb980e2f85490232d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.077715 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956b5e05bda648349ba26d5c1ea3b279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.078563 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 5f57568e1c4c4573ac18d9f621eeb4c7 (DEFAULT_TABLE table=default.Table_1510db6bb88343d0aa4e95a01ecf17a6 [id=ad11a4d7ee734c9bab8278aebac5cb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.078868 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f57568e1c4c4573ac18d9f621eeb4c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.082371 753 tablet_service.cc:1511] Processing CreateTablet for tablet 1765a95cc3414c73b7e3918913a35937 (DEFAULT_TABLE table=default.Table_468330cd933549a587255653023d8e49 [id=13ca7143dd0249d597c79bbfc17ae7f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.082751 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1765a95cc3414c73b7e3918913a35937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.085891 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 222c41e6cd8b4d0093fd692618efb107
I20260505 07:53:54.087013 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 222c41e6cd8b4d0093fd692618efb107 with delete_type TABLET_DATA_DELETED (replaced by tablet b404e50ae092419c9396772b3b14eeaa) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.088204 754 tablet_service.cc:1511] Processing CreateTablet for tablet 5dab02a546b944f799045723a515118f (DEFAULT_TABLE table=default.Table_448efd98249942379556e125b25d169c [id=b2adaf7422c7403c8b331c0d33f0ad53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.088287 595 catalog_manager.cc:6741] ReplaceTablet: tablet 222c41e6cd8b4d0093fd692618efb107 deleted and replaced by tablet b404e50ae092419c9396772b3b14eeaa
I20260505 07:53:54.088544 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dab02a546b944f799045723a515118f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.090143 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
/tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:33565
--webserver_interface=127.0.105.61
--webserver_port=43503
--builtin_ntp_servers=127.0.105.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260505 07:53:54.098742 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_14478c9067fb42d6a508110c0aa360ed" } new_table_name: "default.Table_8e821d63b81c43258a16db52d5295ad2" modify_external_catalogs: true
I20260505 07:53:54.100788 754 tablet_service.cc:1511] Processing CreateTablet for tablet 2782cf753bc44bf4a102dd785ea3c6b8 (DEFAULT_TABLE table=default.Table_6522a65a3ea840658d0e343d76a64379 [id=37397b87cfe445f2adfb7eb0af27454b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.101224 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2782cf753bc44bf4a102dd785ea3c6b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.104394 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_9e1b44e1d482455e8cc700e5de6c4a51"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.105139 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.107211 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_156bb9f5bc904dba8a29f1496abf6310"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.107846 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_156bb9f5bc904dba8a29f1496abf6310 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.108211 757 tablet_service.cc:1511] Processing CreateTablet for tablet 956b5e05bda648349ba26d5c1ea3b279 (DEFAULT_TABLE table=default.Table_74bdaa65d97949c9ae16b85c62dbcc59 [id=d524458d873a48cb980e2f85490232d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.108568 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956b5e05bda648349ba26d5c1ea3b279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.109040 596 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b6b237d0b9e54608b6c7b015becc676d" } new_table_name: "default.Table_a951568d1c5845d9a5329bd1654b99db" modify_external_catalogs: true
I20260505 07:53:54.110332 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f4c4ec691a3445b08b83d566bd437fb8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.110852 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4c4ec691a3445b08b83d566bd437fb8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.114676 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.115914 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 222c41e6cd8b4d0093fd692618efb107 with delete_type TABLET_DATA_DELETED (replaced by tablet b404e50ae092419c9396772b3b14eeaa) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.116255 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 6a2b859a5dd54bc8b36f3197b58afcea (table default.Table_31dd1382f8b4494e9e13ac5b3ac18aa0 [id=b8e0010041cc477d9ce226cca5a28a52]) successfully deleted
I20260505 07:53:54.003502 900 tablet_service.cc:1511] Processing CreateTablet for tablet 3781d0bbcd1b4189bd836eeae4236f40 (DEFAULT_TABLE table=default.Table_14478c9067fb42d6a508110c0aa360ed [id=b4934f65dfeb4a679111783301bb1fb9]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:54.116739 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 31b151ff45804fbaa76cfbec86c6fd7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31b151ff45804fbaa76cfbec86c6fd7a already in progress: creating tablet
I20260505 07:53:54.116799 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3781d0bbcd1b4189bd836eeae4236f40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.117030 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 070d1b1b30d447539911349841638c12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.016752 901 tablet_service.cc:1511] Processing CreateTablet for tablet 56cbbb89f6b84407b2c344ec62402b03 (DEFAULT_TABLE table=default.Table_da224a28c7e8451a8f4c04fcd2160c6b [id=bd575286f4e94df3b335b5e79d8f971c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.119855 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56cbbb89f6b84407b2c344ec62402b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.121338 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1de414720ea44bca392e13c626284bc" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:54.122290 988 leader_election.cc:343] T e1de414720ea44bca392e13c626284bc P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:53:54.123490 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3781d0bbcd1b4189bd836eeae4236f40 consensus state has no leader
I20260505 07:53:54.019539 895 tablet_service.cc:1511] Processing CreateTablet for tablet 5f57568e1c4c4573ac18d9f621eeb4c7 (DEFAULT_TABLE table=default.Table_1510db6bb88343d0aa4e95a01ecf17a6 [id=ad11a4d7ee734c9bab8278aebac5cb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.123927 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5f57568e1c4c4573ac18d9f621eeb4c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.024145 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 56a783ccd69d453ea7e7c004bf55170e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.028697 1172 tablet_bootstrap.cc:492] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.127895 1172 ts_tablet_manager.cc:1403] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.249s user 0.001s sys 0.004s
W20260505 07:53:54.128047 596 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b404e50ae092419c9396772b3b14eeaa has no consensus state
I20260505 07:53:54.128577 1172 raft_consensus.cc:359] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.128769 1172 raft_consensus.cc:385] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.128837 1172 raft_consensus.cc:740] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.129058 1172 consensus_queue.cc:260] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.129576 1172 ts_tablet_manager.cc:1434] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:53:54.038908 897 tablet_service.cc:1511] Processing CreateTablet for tablet 1765a95cc3414c73b7e3918913a35937 (DEFAULT_TABLE table=default.Table_468330cd933549a587255653023d8e49 [id=13ca7143dd0249d597c79bbfc17ae7f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.039474 1171 ts_tablet_manager.cc:1950] T b705bde46048477b92436eea59cde986 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.050422 899 tablet_service.cc:1511] Processing CreateTablet for tablet 5dab02a546b944f799045723a515118f (DEFAULT_TABLE table=default.Table_448efd98249942379556e125b25d169c [id=b2adaf7422c7403c8b331c0d33f0ad53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.056408 898 tablet_service.cc:1511] Processing CreateTablet for tablet 2782cf753bc44bf4a102dd785ea3c6b8 (DEFAULT_TABLE table=default.Table_6522a65a3ea840658d0e343d76a64379 [id=37397b87cfe445f2adfb7eb0af27454b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.067014 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.067647 893 tablet_service.cc:1511] Processing CreateTablet for tablet 956b5e05bda648349ba26d5c1ea3b279 (DEFAULT_TABLE table=default.Table_74bdaa65d97949c9ae16b85c62dbcc59 [id=d524458d873a48cb980e2f85490232d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.087453 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 222c41e6cd8b4d0093fd692618efb107 with delete_type TABLET_DATA_DELETED (replaced by tablet b404e50ae092419c9396772b3b14eeaa) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.130939 1172 tablet_bootstrap.cc:492] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.133013 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 956b5e05bda648349ba26d5c1ea3b279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.135882 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d1ecbab8fbf047cf847ebbb4aeee4b63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.136170 1172 tablet_bootstrap.cc:654] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.136296 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_c31ef73b05d44d77b065f8f92c390efb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.136354 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7af69942bd9b48e79bc11f90b8fd656c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.136586 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_464b120ca20e45fab0404529a41c17ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.137193 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_70e3149cee264c889c7677c3757ef266"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.138092 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_156bb9f5bc904dba8a29f1496abf6310" } new_table_name: "default.Table_da8a2a5c54bf40a784a8a01b7245b280" modify_external_catalogs: true
I20260505 07:53:54.141012 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dab02a546b944f799045723a515118f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.142108 1221 raft_consensus.cc:493] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:54.142246 1221 raft_consensus.cc:515] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:54.143052 1221 leader_election.cc:290] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:54.143846 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2782cf753bc44bf4a102dd785ea3c6b8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:54.144116 718 leader_election.cc:343] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 070d1b1b30d447539911349841638c12
W20260505 07:53:54.146570 717 leader_election.cc:343] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 070d1b1b30d447539911349841638c12
I20260505 07:53:54.146749 717 leader_election.cc:304] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:54.147130 1221 raft_consensus.cc:2749] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:53:54.147701 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7af69942bd9b48e79bc11f90b8fd656c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.148857 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1765a95cc3414c73b7e3918913a35937. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:54.149008 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c31ef73b05d44d77b065f8f92c390efb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:54.150007 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_464b120ca20e45fab0404529a41c17ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.150665 1229 leader_election.cc:290] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:54.150977 1178 tablet_replica.cc:333] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
W20260505 07:53:54.151254 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_70e3149cee264c889c7677c3757ef266 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.151270 1178 raft_consensus.cc:2243] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.151738 1178 raft_consensus.cc:2272] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.151700 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70dedbd58ca402daf0fdde68cc5b574" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:53:54.151907 782 raft_consensus.cc:3060] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:54.152351 1178 ts_tablet_manager.cc:1916] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:54.153209 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1ecbab8fbf047cf847ebbb4aeee4b63 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.154322 1171 tablet_replica.cc:333] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.154507 1171 raft_consensus.cc:2243] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.154529 782 raft_consensus.cc:2468] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:53:54.154649 1171 raft_consensus.cc:2272] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.154924 1176 tablet_bootstrap.cc:492] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:54.155059 1176 ts_tablet_manager.cc:1403] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.082s user 0.000s sys 0.003s
I20260505 07:53:54.155889 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet b705bde46048477b92436eea59cde986 (table default.Table_cd6f1e7c73784523be3f560bd458d7b2 [id=caadebc30e5a4c39877cd29df1e35306]) successfully deleted
I20260505 07:53:54.155877 1176 raft_consensus.cc:359] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.156037 1176 raft_consensus.cc:385] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.156118 1176 raft_consensus.cc:740] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
W20260505 07:53:54.156150 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 31b151ff45804fbaa76cfbec86c6fd7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31b151ff45804fbaa76cfbec86c6fd7a already in progress: creating tablet
I20260505 07:53:54.156322 1176 consensus_queue.cc:260] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.156436 854 leader_election.cc:304] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:53:54.156848 1176 ts_tablet_manager.cc:1434] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:53:54.157034 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c70dedbd58ca402daf0fdde68cc5b574" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:53:54.157506 856 leader_election.cc:343] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.158138 1171 ts_tablet_manager.cc:1916] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.160303 1181 tablet_bootstrap.cc:492] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:54.160470 1181 ts_tablet_manager.cc:1403] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.230s user 0.025s sys 0.042s
I20260505 07:53:54.161172 1181 raft_consensus.cc:359] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.161369 1181 raft_consensus.cc:385] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.161473 1181 raft_consensus.cc:740] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.162493 1181 consensus_queue.cc:260] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.163895 1171 ts_tablet_manager.cc:1929] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.164093 1171 log.cc:1199] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/6a2b859a5dd54bc8b36f3197b58afcea
I20260505 07:53:54.164580 1171 ts_tablet_manager.cc:1950] T 6a2b859a5dd54bc8b36f3197b58afcea P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.165313 1172 tablet_bootstrap.cc:492] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.165509 1172 ts_tablet_manager.cc:1403] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.003s
W20260505 07:53:54.166182 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet faf1140fe05e4f9493563ab8464c582b has no consensus state
I20260505 07:53:54.166287 1172 raft_consensus.cc:359] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.166540 1172 raft_consensus.cc:385] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.166715 1172 raft_consensus.cc:740] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.167021 1172 consensus_queue.cc:260] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.167541 1181 ts_tablet_manager.cc:1434] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:53:54.167533 1172 ts_tablet_manager.cc:1434] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:53:54.168426 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_38239693c58a4c8095e38c507eaf96ec" } new_table_name: "default.Table_4bd82b12eff3408eaa6b8b4d7413e3f5" modify_external_catalogs: true
I20260505 07:53:54.168700 598 master_service.cc:954] ReplaceTablet: received request to replace tablet 05b8c0a2185e473cbb4165fed9e94c8a from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:54.169313 1180 raft_consensus.cc:2804] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:54.169340 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6522a65a3ea840658d0e343d76a64379" } new_table_name: "default.Table_66d69d43d5d24b9b88a4d885b0ce8f1d" modify_external_catalogs: true
I20260505 07:53:54.169485 1180 raft_consensus.cc:697] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:53:54.169950 1180 consensus_queue.cc:237] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.168522 1172 tablet_bootstrap.cc:492] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.170781 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ba04ccad67434c7884329b23efea4bfd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.171255 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba04ccad67434c7884329b23efea4bfd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.171422 1181 tablet_bootstrap.cc:492] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:54.172557 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_eb39510acd7a40b6a7f20bd09e300ff7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.172776 1172 tablet_bootstrap.cc:654] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:54.172971 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb39510acd7a40b6a7f20bd09e300ff7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.173259 1181 tablet_bootstrap.cc:654] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.174333 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 6a2b859a5dd54bc8b36f3197b58afcea (table default.Table_31dd1382f8b4494e9e13ac5b3ac18aa0 [id=b8e0010041cc477d9ce226cca5a28a52]) successfully deleted
W20260505 07:53:54.175079 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 31b151ff45804fbaa76cfbec86c6fd7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 31b151ff45804fbaa76cfbec86c6fd7a already in progress: creating tablet
I20260505 07:53:54.175304 1175 ts_tablet_manager.cc:1916] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:54.176095 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet edb76b7961844443a9b41353c2d77d5a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edb76b7961844443a9b41353c2d77d5a already in progress: creating tablet
I20260505 07:53:54.176411 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 6a2b859a5dd54bc8b36f3197b58afcea with delete_type TABLET_DATA_DELETED (replaced by tablet 8ca4cf248adb4f768c5586d72614ad42) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:54.177057 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 05b8c0a2185e473cbb4165fed9e94c8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05b8c0a2185e473cbb4165fed9e94c8a already in progress: creating tablet
W20260505 07:53:54.177992 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 370be7a513ee49028c8d0662b43c8d91 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 370be7a513ee49028c8d0662b43c8d91 already in progress: creating tablet
I20260505 07:53:54.178495 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 772a65f4016843d18fbca25842f76de0 from {username='slave'} at 127.0.0.1:47238
W20260505 07:53:54.179075 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet cf95f4331e4540ae84e329cf577427b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf95f4331e4540ae84e329cf577427b8 already in progress: creating tablet
I20260505 07:53:54.179780 1171 tablet_replica.cc:333] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.181790 1176 tablet_bootstrap.cc:492] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:54.182586 600 master_service.cc:954] ReplaceTablet: received request to replace tablet b7cce14283e7405690a15e26be8a45b3 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:54.186966 1176 tablet_bootstrap.cc:654] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.186954 1171 raft_consensus.cc:2243] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:53:54.189445 1171 pending_rounds.cc:70] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:53:54.189656 1171 pending_rounds.cc:77] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555429562617856 op_type: NO_OP noop_request { }
I20260505 07:53:54.189908 1171 raft_consensus.cc:2889] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:53:54.190037 1171 raft_consensus.cc:2272] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.190905 1172 tablet_bootstrap.cc:492] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.191051 1172 ts_tablet_manager.cc:1403] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.000s
I20260505 07:53:54.193329 594 master_service.cc:954] ReplaceTablet: received request to replace tablet cf95f4331e4540ae84e329cf577427b8 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:54.194195 1181 tablet_bootstrap.cc:492] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:54.194341 1181 ts_tablet_manager.cc:1403] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.001s
I20260505 07:53:54.194623 1171 ts_tablet_manager.cc:1916] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.195087 1181 raft_consensus.cc:359] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.195267 1181 raft_consensus.cc:385] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.195425 1181 raft_consensus.cc:740] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.195667 1181 consensus_queue.cc:260] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.197109 753 tablet_service.cc:1558] Processing DeleteTablet for tablet a25715f0fa504e92b54c48712c2c2f48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.197467 1172 raft_consensus.cc:359] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.197654 1172 raft_consensus.cc:385] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.197733 1172 raft_consensus.cc:740] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.198012 1172 consensus_queue.cc:260] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.198916 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 8087cab2da594607a0c3f7cf5839c6f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.200390 1181 ts_tablet_manager.cc:1434] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:53:54.201099 1178 ts_tablet_manager.cc:1929] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.202826 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet f296602a34634c649d2573cb178b7053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.204357 1181 tablet_bootstrap.cc:492] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:54.204589 1172 ts_tablet_manager.cc:1434] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260505 07:53:54.205466 1172 tablet_bootstrap.cc:492] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.207190 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 8087cab2da594607a0c3f7cf5839c6f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.207903 1172 tablet_bootstrap.cc:654] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:54.209939 717 consensus_peers.cc:597] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:53:54.212473 921 raft_consensus.cc:3060] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:53:54.214252 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5de9e0fe8eb7477ab1cfa64b808575d8 consensus state has no leader
W20260505 07:53:54.215059 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2782cf753bc44bf4a102dd785ea3c6b8 consensus state has no leader
I20260505 07:53:54.216930 921 raft_consensus.cc:1275] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Refusing update from remote peer eac7e1e8caba43cdb4df80cbb80e891c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260505 07:53:54.218035 1171 ts_tablet_manager.cc:1929] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:53:54.217947 1179 consensus_queue.cc:1048] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:53:54.218165 1171 log.cc:1199] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/c70dedbd58ca402daf0fdde68cc5b574
I20260505 07:53:54.218546 1171 ts_tablet_manager.cc:1950] T c70dedbd58ca402daf0fdde68cc5b574 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.219493 1171 tablet_replica.cc:333] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.219571 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet c70dedbd58ca402daf0fdde68cc5b574 (table default.Table_2dba7ce60556464488c3d723065cbbfd [id=b75fe4dac469427c8cf8ccd447f70973]) successfully deleted
I20260505 07:53:54.219700 1171 raft_consensus.cc:2243] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:53:54.219935 1171 pending_rounds.cc:70] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:53:54.223037 1178 log.cc:1199] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/56a783ccd69d453ea7e7c004bf55170e
I20260505 07:53:54.223426 1178 ts_tablet_manager.cc:1950] T 56a783ccd69d453ea7e7c004bf55170e P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:54.223553 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 8573f8bd424641438e2aa6ff6869180e (DEFAULT_TABLE table=default.Table_7d1be5262b41414f8a0fc37fba1cc28a [id=128f12d4ca8c43f6887ae3240610ef60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.223956 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8573f8bd424641438e2aa6ff6869180e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.225169 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 451e99a0dfec4823812e5991147b132e (DEFAULT_TABLE table=default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.225523 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 451e99a0dfec4823812e5991147b132e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.226521 1181 tablet_bootstrap.cc:654] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.228856 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 56a783ccd69d453ea7e7c004bf55170e (table default.Table_efa325aff89944cc9d011916a21d132e [id=dda64712b2cd4f1286abaf5c08022180]) successfully deleted
I20260505 07:53:54.230098 600 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet b7cce14283e7405690a15e26be8a45b3
I20260505 07:53:54.230183 760 tablet_service.cc:1511] Processing CreateTablet for tablet 8573f8bd424641438e2aa6ff6869180e (DEFAULT_TABLE table=default.Table_7d1be5262b41414f8a0fc37fba1cc28a [id=128f12d4ca8c43f6887ae3240610ef60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.230657 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8573f8bd424641438e2aa6ff6869180e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.231137 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6e9d636b8b314e5b888c725d34775180" } modify_external_catalogs: true
I20260505 07:53:54.231283 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6e9d636b8b314e5b888c725d34775180" } modify_external_catalogs: true
I20260505 07:53:54.231711 600 catalog_manager.cc:6741] ReplaceTablet: tablet b7cce14283e7405690a15e26be8a45b3 deleted and replaced by tablet 6429919aba17447dbdb393176360bbb0
I20260505 07:53:54.232368 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4805d11726a3446b86b7a990c0dbb3c5" } modify_external_catalogs: true
I20260505 07:53:54.232542 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4805d11726a3446b86b7a990c0dbb3c5" } modify_external_catalogs: true
I20260505 07:53:54.237604 754 tablet_service.cc:1558] Processing DeleteTablet for tablet b7cce14283e7405690a15e26be8a45b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 6429919aba17447dbdb393176360bbb0) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.221182 1229 raft_consensus.cc:493] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:54.237911 1229 raft_consensus.cc:515] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.238183 1176 tablet_bootstrap.cc:492] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:54.238351 1176 ts_tablet_manager.cc:1403] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.057s user 0.004s sys 0.000s
I20260505 07:53:54.238540 1229 leader_election.cc:290] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:54.239125 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f296602a34634c649d2573cb178b7053" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:54.239195 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 772a65f4016843d18fbca25842f76de0
I20260505 07:53:54.239203 1176 raft_consensus.cc:359] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.239333 782 raft_consensus.cc:2468] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:53:54.239387 1176 raft_consensus.cc:385] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.239449 1176 raft_consensus.cc:740] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.224565 901 tablet_service.cc:1511] Processing CreateTablet for tablet 8573f8bd424641438e2aa6ff6869180e (DEFAULT_TABLE table=default.Table_7d1be5262b41414f8a0fc37fba1cc28a [id=128f12d4ca8c43f6887ae3240610ef60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.239655 595 catalog_manager.cc:6741] ReplaceTablet: tablet 772a65f4016843d18fbca25842f76de0 deleted and replaced by tablet 1776a12afc434d639cf98a9d253ef2d6
I20260505 07:53:54.239894 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8573f8bd424641438e2aa6ff6869180e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.240634 1175 ts_tablet_manager.cc:1929] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.240788 1175 log.cc:1199] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/e1de414720ea44bca392e13c626284bc
I20260505 07:53:54.241137 1175 ts_tablet_manager.cc:1950] T e1de414720ea44bca392e13c626284bc P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:54.242074 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet e1de414720ea44bca392e13c626284bc (table default.Table_1cae231a7d5647bf887cd8dd8bb0ded9 [id=7ba2bfd0de87417c944ea8a4edf99b14]) successfully deleted
I20260505 07:53:54.242900 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet c7652191b1d44ba59d856a2b0d7017b1
I20260505 07:53:54.242960 760 tablet_service.cc:1511] Processing CreateTablet for tablet e1a9420a81844d2092381123d9297661 (DEFAULT_TABLE table=default.Table_476efdff9cee417b941d0796953265f5 [id=0d8de70d77c54c2491b257d09c8474c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.243327 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a9420a81844d2092381123d9297661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.232261 893 tablet_service.cc:1558] Processing DeleteTablet for tablet b7cce14283e7405690a15e26be8a45b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 6429919aba17447dbdb393176360bbb0) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.233681 895 tablet_service.cc:1511] Processing CreateTablet for tablet 451e99a0dfec4823812e5991147b132e (DEFAULT_TABLE table=default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.243952 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 451e99a0dfec4823812e5991147b132e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.243958 751 tablet_service.cc:1558] Processing DeleteTablet for tablet c7652191b1d44ba59d856a2b0d7017b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.245939 598 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 1776a12afc434d639cf98a9d253ef2d6
I20260505 07:53:54.234475 898 tablet_service.cc:1511] Processing CreateTablet for tablet e1a9420a81844d2092381123d9297661 (DEFAULT_TABLE table=default.Table_476efdff9cee417b941d0796953265f5 [id=0d8de70d77c54c2491b257d09c8474c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.235272 897 tablet_service.cc:1511] Processing CreateTablet for tablet c7652191b1d44ba59d856a2b0d7017b1 (DEFAULT_TABLE table=default.Table_6e9d636b8b314e5b888c725d34775180 [id=73c0bcbf3485405589f58b8175bfc3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.246942 1176 consensus_queue.cc:260] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.247087 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7652191b1d44ba59d856a2b0d7017b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.247047 854 leader_election.cc:304] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:53:54.247357 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 772a65f4016843d18fbca25842f76de0 with delete_type TABLET_DATA_DELETED (replaced by tablet 1776a12afc434d639cf98a9d253ef2d6) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.247545 1229 raft_consensus.cc:2804] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:54.247687 1229 raft_consensus.cc:493] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:54.247795 1229 raft_consensus.cc:3060] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:54.248225 899 tablet_service.cc:1558] Processing DeleteTablet for tablet c7652191b1d44ba59d856a2b0d7017b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.248775 753 tablet_service.cc:1511] Processing CreateTablet for tablet c7652191b1d44ba59d856a2b0d7017b1 (DEFAULT_TABLE table=default.Table_6e9d636b8b314e5b888c725d34775180 [id=73c0bcbf3485405589f58b8175bfc3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.248945 893 tablet_service.cc:1558] Processing DeleteTablet for tablet 772a65f4016843d18fbca25842f76de0 with delete_type TABLET_DATA_DELETED (replaced by tablet 1776a12afc434d639cf98a9d253ef2d6) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.249159 754 tablet_service.cc:1511] Processing CreateTablet for tablet 451e99a0dfec4823812e5991147b132e (DEFAULT_TABLE table=default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.249802 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7652191b1d44ba59d856a2b0d7017b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.251533 1229 raft_consensus.cc:515] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.251958 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a9420a81844d2092381123d9297661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.252514 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f296602a34634c649d2573cb178b7053" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:53:54.252744 782 raft_consensus.cc:3060] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:54.252743 1171 pending_rounds.cc:77] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555428752056320 op_type: NO_OP noop_request { }
I20260505 07:53:54.252801 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 451e99a0dfec4823812e5991147b132e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.252876 1171 raft_consensus.cc:2889] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:53:54.252997 1171 raft_consensus.cc:2272] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.255574 782 raft_consensus.cc:2468] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:53:54.256307 854 leader_election.cc:304] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:53:54.256731 1180 raft_consensus.cc:2804] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:54.256870 1180 raft_consensus.cc:697] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:53:54.257159 1180 consensus_queue.cc:237] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.262814 1229 leader_election.cc:290] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:54.265595 1176 ts_tablet_manager.cc:1434] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.027s user 0.001s sys 0.000s
I20260505 07:53:54.269763 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7d1be5262b41414f8a0fc37fba1cc28a" } modify_external_catalogs: true
I20260505 07:53:54.269975 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7d1be5262b41414f8a0fc37fba1cc28a" } modify_external_catalogs: true
I20260505 07:53:54.270303 1171 ts_tablet_manager.cc:1916] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.271162 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_aa53055d0748464697a41feba3c4f036" } modify_external_catalogs: true
I20260505 07:53:54.271350 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_aa53055d0748464697a41feba3c4f036" } modify_external_catalogs: true
I20260505 07:53:54.273377 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_da8a2a5c54bf40a784a8a01b7245b280" } modify_external_catalogs: true
I20260505 07:53:54.273253 1247 tablet.cc:1724] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:53:54.273599 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_da8a2a5c54bf40a784a8a01b7245b280" } modify_external_catalogs: true
I20260505 07:53:54.274972 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.283134 1026 tablet_service.cc:1511] Processing CreateTablet for tablet e1a9420a81844d2092381123d9297661 (DEFAULT_TABLE table=default.Table_476efdff9cee417b941d0796953265f5 [id=0d8de70d77c54c2491b257d09c8474c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.283574 1029 tablet_service.cc:1511] Processing CreateTablet for tablet c7652191b1d44ba59d856a2b0d7017b1 (DEFAULT_TABLE table=default.Table_6e9d636b8b314e5b888c725d34775180 [id=73c0bcbf3485405589f58b8175bfc3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.283824 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 772a65f4016843d18fbca25842f76de0 with delete_type TABLET_DATA_DELETED (replaced by tablet 1776a12afc434d639cf98a9d253ef2d6) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.284060 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet c7652191b1d44ba59d856a2b0d7017b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.286278 1033 tablet_service.cc:1558] Processing DeleteTablet for tablet b7cce14283e7405690a15e26be8a45b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 6429919aba17447dbdb393176360bbb0) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.288759 1248 tablet.cc:1724] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:53:54.289142 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f296602a34634c649d2573cb178b7053" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:54.289371 1055 raft_consensus.cc:2468] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:53:54.290055 1176 tablet_bootstrap.cc:492] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:54.290349 1172 tablet_bootstrap.cc:492] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.290469 1172 ts_tablet_manager.cc:1403] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.085s user 0.012s sys 0.006s
I20260505 07:53:54.291723 1172 raft_consensus.cc:359] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.291981 1172 raft_consensus.cc:385] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.292065 1172 raft_consensus.cc:740] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.292312 1176 tablet_bootstrap.cc:654] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.292313 1172 consensus_queue.cc:260] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.292882 1172 ts_tablet_manager.cc:1434] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:53:54.294430 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f296602a34634c649d2573cb178b7053" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:53:54.294615 1052 raft_consensus.cc:3060] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:54.294922 1172 tablet_bootstrap.cc:492] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.296181 1171 ts_tablet_manager.cc:1929] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:53:54.296324 1171 log.cc:1199] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e1de414720ea44bca392e13c626284bc
I20260505 07:53:54.296777 1171 ts_tablet_manager.cc:1950] T e1de414720ea44bca392e13c626284bc P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.297241 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e1de414720ea44bca392e13c626284bc (table default.Table_1cae231a7d5647bf887cd8dd8bb0ded9 [id=7ba2bfd0de87417c944ea8a4edf99b14]) successfully deleted
I20260505 07:53:54.297289 1052 raft_consensus.cc:2468] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
W20260505 07:53:54.298466 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet edb76b7961844443a9b41353c2d77d5a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet edb76b7961844443a9b41353c2d77d5a already in progress: creating tablet
I20260505 07:53:54.299189 1220 raft_consensus.cc:493] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:53:54.299407 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 4d6490f568024510ab843ec5205a4d1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d6490f568024510ab843ec5205a4d1a already in progress: creating tablet
I20260505 07:53:54.299347 1220 raft_consensus.cc:515] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.299705 1171 tablet_replica.cc:333] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.299854 1171 raft_consensus.cc:2243] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:53:54.300082 1171 pending_rounds.cc:70] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:53:54.300132 1171 pending_rounds.cc:77] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555429922115584 op_type: NO_OP noop_request { }
I20260505 07:53:54.301939 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7652191b1d44ba59d856a2b0d7017b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.302083 1171 raft_consensus.cc:2889] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:53:54.302222 1171 raft_consensus.cc:2272] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.302778 1172 tablet_bootstrap.cc:654] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.304240 1171 ts_tablet_manager.cc:1916] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.300071 1220 leader_election.cc:290] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:54.311432 1176 tablet_bootstrap.cc:492] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
W20260505 07:53:54.311594 718 leader_election.cc:343] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 0fd990e80a90418dbb9b94cb6e2ee156
I20260505 07:53:54.311637 1176 ts_tablet_manager.cc:1403] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.002s
I20260505 07:53:54.311758 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fd990e80a90418dbb9b94cb6e2ee156" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:54.312294 717 leader_election.cc:343] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.312446 717 leader_election.cc:304] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:54.312517 1172 tablet_bootstrap.cc:492] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.312665 1172 ts_tablet_manager.cc:1403] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.003s
I20260505 07:53:54.312747 1176 raft_consensus.cc:359] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.312880 1220 raft_consensus.cc:2749] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:54.312924 1176 raft_consensus.cc:385] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.313033 1176 raft_consensus.cc:740] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.313275 1176 consensus_queue.cc:260] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.313422 1172 raft_consensus.cc:359] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.313606 1172 raft_consensus.cc:385] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.313673 1172 raft_consensus.cc:740] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.313921 1172 consensus_queue.cc:260] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.314705 1172 ts_tablet_manager.cc:1434] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:53:54.314812 1176 ts_tablet_manager.cc:1434] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:54.315027 1172 tablet_bootstrap.cc:492] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.316112 1171 ts_tablet_manager.cc:1929] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:53:54.316318 1171 log.cc:1199] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/f296602a34634c649d2573cb178b7053
I20260505 07:53:54.316579 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a9420a81844d2092381123d9297661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.316808 1171 ts_tablet_manager.cc:1950] T f296602a34634c649d2573cb178b7053 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.317520 1171 tablet_replica.cc:333] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.317687 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet f296602a34634c649d2573cb178b7053 (table default.Table_b3f0f5990c0946598233d0bd2fe0084d [id=db183a96c4214053beda00a79bdd6b44]) successfully deleted
W20260505 07:53:54.317946 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 5b124eda66364b7a81426934b98bca48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5b124eda66364b7a81426934b98bca48 already in progress: creating tablet
W20260505 07:53:54.318154 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 2850a301ac4f435a96742b00bf54baa9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2850a301ac4f435a96742b00bf54baa9 already in progress: creating tablet
I20260505 07:53:54.319105 898 tablet_service.cc:1511] Processing CreateTablet for tablet c6dcbebddcdc4ad9a788c01134e1101c (DEFAULT_TABLE table=default.Table_ba04ccad67434c7884329b23efea4bfd [id=36cc8dc1683b4ef194ec299e4fed4de9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.319325 754 tablet_service.cc:1511] Processing CreateTablet for tablet c6dcbebddcdc4ad9a788c01134e1101c (DEFAULT_TABLE table=default.Table_ba04ccad67434c7884329b23efea4bfd [id=36cc8dc1683b4ef194ec299e4fed4de9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.319417 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dcbebddcdc4ad9a788c01134e1101c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.320940 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 9862b87d74974dc49d7dc9049b2a694a (DEFAULT_TABLE table=default.Table_d1ecbab8fbf047cf847ebbb4aeee4b63 [id=164a806dd1a9450ea55b4ea1be0caa64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.321326 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9862b87d74974dc49d7dc9049b2a694a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.323880 759 tablet_service.cc:1511] Processing CreateTablet for tablet 9862b87d74974dc49d7dc9049b2a694a (DEFAULT_TABLE table=default.Table_d1ecbab8fbf047cf847ebbb4aeee4b63 [id=164a806dd1a9450ea55b4ea1be0caa64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.324896 1031 tablet_service.cc:1511] Processing CreateTablet for tablet e85b44337f62494c997c0008fc492aab (DEFAULT_TABLE table=default.Table_7af69942bd9b48e79bc11f90b8fd656c [id=06480a024264477596ab9058373c273b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.325074 893 tablet_service.cc:1511] Processing CreateTablet for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d (DEFAULT_TABLE table=default.Table_f4c4ec691a3445b08b83d566bd437fb8 [id=740016f529e141b58f3ea08e9b984052]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.325227 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e85b44337f62494c997c0008fc492aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.325150 895 tablet_service.cc:1511] Processing CreateTablet for tablet 9862b87d74974dc49d7dc9049b2a694a (DEFAULT_TABLE table=default.Table_d1ecbab8fbf047cf847ebbb4aeee4b63 [id=164a806dd1a9450ea55b4ea1be0caa64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.325408 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.325666 901 tablet_service.cc:1511] Processing CreateTablet for tablet 3272555a8c814c0ebe61ca32d715212b (DEFAULT_TABLE table=default.Table_70e3149cee264c889c7677c3757ef266 [id=27e306dc7bda4ca0b28f72e8636b91d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.326026 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3272555a8c814c0ebe61ca32d715212b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.326095 897 tablet_service.cc:1511] Processing CreateTablet for tablet 25a74b65d5d6459293ab4461f4d34cf2 (DEFAULT_TABLE table=default.Table_eb39510acd7a40b6a7f20bd09e300ff7 [id=94a6d460807e43768c78bc319239b699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.326416 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a74b65d5d6459293ab4461f4d34cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.327246 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 3ac06bb9d273457ead843f5dd754539c (DEFAULT_TABLE table=default.Table_c31ef73b05d44d77b065f8f92c390efb [id=4c5b3219388f43d4866a6ef3b32bf603]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.327633 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ac06bb9d273457ead843f5dd754539c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.327904 900 tablet_service.cc:1511] Processing CreateTablet for tablet faf1140fe05e4f9493563ab8464c582b (DEFAULT_TABLE table=default.Table_da8a2a5c54bf40a784a8a01b7245b280 [id=2170bb22bfb747c5b4f24ba83810f89e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.328274 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet faf1140fe05e4f9493563ab8464c582b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.330848 899 tablet_service.cc:1511] Processing CreateTablet for tablet e85b44337f62494c997c0008fc492aab (DEFAULT_TABLE table=default.Table_7af69942bd9b48e79bc11f90b8fd656c [id=06480a024264477596ab9058373c273b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.331199 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e85b44337f62494c997c0008fc492aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.331845 1035 tablet_service.cc:1511] Processing CreateTablet for tablet faf1140fe05e4f9493563ab8464c582b (DEFAULT_TABLE table=default.Table_da8a2a5c54bf40a784a8a01b7245b280 [id=2170bb22bfb747c5b4f24ba83810f89e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.332140 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet faf1140fe05e4f9493563ab8464c582b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.334870 751 tablet_service.cc:1511] Processing CreateTablet for tablet 3272555a8c814c0ebe61ca32d715212b (DEFAULT_TABLE table=default.Table_70e3149cee264c889c7677c3757ef266 [id=27e306dc7bda4ca0b28f72e8636b91d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.335246 758 tablet_service.cc:1511] Processing CreateTablet for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d (DEFAULT_TABLE table=default.Table_f4c4ec691a3445b08b83d566bd437fb8 [id=740016f529e141b58f3ea08e9b984052]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.335573 757 tablet_service.cc:1511] Processing CreateTablet for tablet e85b44337f62494c997c0008fc492aab (DEFAULT_TABLE table=default.Table_7af69942bd9b48e79bc11f90b8fd656c [id=06480a024264477596ab9058373c273b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.334893 756 tablet_service.cc:1511] Processing CreateTablet for tablet faf1140fe05e4f9493563ab8464c582b (DEFAULT_TABLE table=default.Table_da8a2a5c54bf40a784a8a01b7245b280 [id=2170bb22bfb747c5b4f24ba83810f89e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.335748 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 25a74b65d5d6459293ab4461f4d34cf2 (DEFAULT_TABLE table=default.Table_eb39510acd7a40b6a7f20bd09e300ff7 [id=94a6d460807e43768c78bc319239b699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.336028 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a74b65d5d6459293ab4461f4d34cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.337164 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9862b87d74974dc49d7dc9049b2a694a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.337921 761 tablet_service.cc:1511] Processing CreateTablet for tablet 3ac06bb9d273457ead843f5dd754539c (DEFAULT_TABLE table=default.Table_c31ef73b05d44d77b065f8f92c390efb [id=4c5b3219388f43d4866a6ef3b32bf603]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.338292 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 3272555a8c814c0ebe61ca32d715212b (DEFAULT_TABLE table=default.Table_70e3149cee264c889c7677c3757ef266 [id=27e306dc7bda4ca0b28f72e8636b91d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.338598 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3272555a8c814c0ebe61ca32d715212b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.343694 755 tablet_service.cc:1511] Processing CreateTablet for tablet 571257bbb7e94315b60d84d6aa4917b1 (DEFAULT_TABLE table=default.Table_464b120ca20e45fab0404529a41c17ec [id=6cb8bc8043474bad9f4278b7eef7c71a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.344081 762 tablet_service.cc:1511] Processing CreateTablet for tablet 32c008bd99224a7595cecc607e5496a9 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.344476 752 tablet_service.cc:1511] Processing CreateTablet for tablet 6429919aba17447dbdb393176360bbb0 (DEFAULT_TABLE table=default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.344852 750 tablet_service.cc:1511] Processing CreateTablet for tablet b404e50ae092419c9396772b3b14eeaa (DEFAULT_TABLE table=default.Table_a951568d1c5845d9a5329bd1654b99db [id=8f533b7aa2374c439f7234a373af31bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.345268 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d (DEFAULT_TABLE table=default.Table_f4c4ec691a3445b08b83d566bd437fb8 [id=740016f529e141b58f3ea08e9b984052]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.345609 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.345775 896 tablet_service.cc:1511] Processing CreateTablet for tablet 571257bbb7e94315b60d84d6aa4917b1 (DEFAULT_TABLE table=default.Table_464b120ca20e45fab0404529a41c17ec [id=6cb8bc8043474bad9f4278b7eef7c71a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.346104 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571257bbb7e94315b60d84d6aa4917b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.346226 897 tablet_service.cc:1511] Processing CreateTablet for tablet 6429919aba17447dbdb393176360bbb0 (DEFAULT_TABLE table=default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.346530 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6429919aba17447dbdb393176360bbb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.346815 1027 tablet_service.cc:1511] Processing CreateTablet for tablet c6dcbebddcdc4ad9a788c01134e1101c (DEFAULT_TABLE table=default.Table_ba04ccad67434c7884329b23efea4bfd [id=36cc8dc1683b4ef194ec299e4fed4de9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.347133 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dcbebddcdc4ad9a788c01134e1101c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.347918 890 tablet_service.cc:1511] Processing CreateTablet for tablet b404e50ae092419c9396772b3b14eeaa (DEFAULT_TABLE table=default.Table_a951568d1c5845d9a5329bd1654b99db [id=8f533b7aa2374c439f7234a373af31bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.348239 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b404e50ae092419c9396772b3b14eeaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.350806 753 tablet_service.cc:1511] Processing CreateTablet for tablet 25a74b65d5d6459293ab4461f4d34cf2 (DEFAULT_TABLE table=default.Table_eb39510acd7a40b6a7f20bd09e300ff7 [id=94a6d460807e43768c78bc319239b699]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.351136 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_464b120ca20e45fab0404529a41c17ec" } new_table_name: "default.Table_477749a4e38f41c48eb5d2fd5a7247e7" modify_external_catalogs: true
I20260505 07:53:54.352186 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_b31d40ee188f46d0bdd481aa88730544"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.352699 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b31d40ee188f46d0bdd481aa88730544 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.354005 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ae34ec4a8b52453dba7158ce065f4798"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.354521 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ae34ec4a8b52453dba7158ce065f4798 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.354740 1178 tablet_replica.cc:333] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:54.354895 1178 raft_consensus.cc:2243] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.355085 1178 raft_consensus.cc:2272] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.351567 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6ff000247d324fe8b55b1398e48cb49a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.356024 1178 ts_tablet_manager.cc:1916] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:54.356125 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ff000247d324fe8b55b1398e48cb49a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:54.359637 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 05b8c0a2185e473cbb4165fed9e94c8a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05b8c0a2185e473cbb4165fed9e94c8a already in progress: creating tablet
I20260505 07:53:54.361847 1178 ts_tablet_manager.cc:1929] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.361990 1178 log.cc:1199] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/6a2b859a5dd54bc8b36f3197b58afcea
I20260505 07:53:54.362388 1178 ts_tablet_manager.cc:1950] T 6a2b859a5dd54bc8b36f3197b58afcea P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:54.363210 1178 tablet_replica.cc:333] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:54.363417 1178 raft_consensus.cc:2243] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.363575 1178 raft_consensus.cc:2272] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.367525 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3272555a8c814c0ebe61ca32d715212b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.371052 1176 tablet_bootstrap.cc:492] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:54.372627 1176 tablet_bootstrap.cc:654] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.372682 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.376449 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c31ef73b05d44d77b065f8f92c390efb" } new_table_name: "default.Table_4e9260f6bf4c4ea3b35c44a98dcf15f1" modify_external_catalogs: true
I20260505 07:53:54.379899 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e85b44337f62494c997c0008fc492aab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.382093 1176 tablet_bootstrap.cc:492] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:54.382242 1176 ts_tablet_manager.cc:1403] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.009s
I20260505 07:53:54.382946 1176 raft_consensus.cc:359] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.383133 1176 raft_consensus.cc:385] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.383209 1176 raft_consensus.cc:740] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.383458 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet faf1140fe05e4f9493563ab8464c582b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.383459 1176 consensus_queue.cc:260] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.385048 1176 ts_tablet_manager.cc:1434] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:54.386076 1176 tablet_bootstrap.cc:492] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:54.387715 1176 tablet_bootstrap.cc:654] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.387804 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dcbebddcdc4ad9a788c01134e1101c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.391829 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9862b87d74974dc49d7dc9049b2a694a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.392190 1175 tablet_replica.cc:333] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:54.392439 1175 raft_consensus.cc:2243] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.392675 1175 raft_consensus.cc:2272] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.393333 1175 ts_tablet_manager.cc:1916] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.395921 1176 tablet_bootstrap.cc:492] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:54.396047 1176 ts_tablet_manager.cc:1403] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260505 07:53:54.396283 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a74b65d5d6459293ab4461f4d34cf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.396908 1176 raft_consensus.cc:359] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.397104 1176 raft_consensus.cc:385] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.397204 1176 raft_consensus.cc:740] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.397447 1176 consensus_queue.cc:260] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.398836 1176 ts_tablet_manager.cc:1434] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260505 07:53:54.399883 1176 tablet_bootstrap.cc:492] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:54.400499 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32c008bd99224a7595cecc607e5496a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.400763 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6429919aba17447dbdb393176360bbb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.401736 1176 tablet_bootstrap.cc:654] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.403911 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ac06bb9d273457ead843f5dd754539c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.407980 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b404e50ae092419c9396772b3b14eeaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.412060 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571257bbb7e94315b60d84d6aa4917b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.416466 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_a951568d1c5845d9a5329bd1654b99db" } new_table_name: "default.Table_45a166ce5441483abbdc62a96ee310c2" modify_external_catalogs: true
I20260505 07:53:54.427948 1175 ts_tablet_manager.cc:1929] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.428105 1175 log.cc:1199] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/689792d9ea734e0bbf1df1c74dab6f83
I20260505 07:53:54.428597 1175 ts_tablet_manager.cc:1950] T 689792d9ea734e0bbf1df1c74dab6f83 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:54.429705 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 689792d9ea734e0bbf1df1c74dab6f83 (table default.Table_8b2e2e36529648cebd2bbdeda19b23b6 [id=0407ce7e0583423e92f51c513897877d]) successfully deleted
W20260505 07:53:54.430485 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 370be7a513ee49028c8d0662b43c8d91 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 370be7a513ee49028c8d0662b43c8d91 already in progress: creating tablet
W20260505 07:53:54.431143 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet cf95f4331e4540ae84e329cf577427b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cf95f4331e4540ae84e329cf577427b8 already in progress: creating tablet
I20260505 07:53:54.431476 1175 tablet_replica.cc:333] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:54.431773 1175 raft_consensus.cc:2243] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.431946 1175 raft_consensus.cc:2272] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.432324 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 8573f8bd424641438e2aa6ff6869180e
I20260505 07:53:54.432458 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 8713b273c7b8438d8749f595ee5f1629
I20260505 07:53:54.432487 1175 ts_tablet_manager.cc:1916] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.433182 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_e9ed38d78117420d98a771675ab512bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.433758 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9ed38d78117420d98a771675ab512bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.435109 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_114b1df99e44400c9d52153e3c154ed1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.435360 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet faf1140fe05e4f9493563ab8464c582b
I20260505 07:53:54.436137 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_4d44519a55e0425d8016d1c0c28ed7f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.436728 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d44519a55e0425d8016d1c0c28ed7f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.437237 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 8573f8bd424641438e2aa6ff6869180e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.437446 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 05b8c0a2185e473cbb4165fed9e94c8a with delete_type TABLET_DATA_DELETED (replaced by tablet 5f57568e1c4c4573ac18d9f621eeb4c7) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:54.435573 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_114b1df99e44400c9d52153e3c154ed1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.437649 753 tablet_service.cc:1558] Processing DeleteTablet for tablet 8713b273c7b8438d8749f595ee5f1629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.437876 762 tablet_service.cc:1558] Processing DeleteTablet for tablet faf1140fe05e4f9493563ab8464c582b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:54.438151 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 571257bbb7e94315b60d84d6aa4917b1 consensus state has no leader
I20260505 07:53:54.446996 1175 ts_tablet_manager.cc:1929] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.447008 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_67a8ca1fae934339bdfe388762b11ab2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.447172 1175 log.cc:1199] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/c70dedbd58ca402daf0fdde68cc5b574
I20260505 07:53:54.447605 1175 ts_tablet_manager.cc:1950] T c70dedbd58ca402daf0fdde68cc5b574 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
W20260505 07:53:54.447652 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67a8ca1fae934339bdfe388762b11ab2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.448822 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet c70dedbd58ca402daf0fdde68cc5b574 (table default.Table_2dba7ce60556464488c3d723065cbbfd [id=b75fe4dac469427c8cf8ccd447f70973]) successfully deleted
W20260505 07:53:54.449640 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 4d6490f568024510ab843ec5205a4d1a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d6490f568024510ab843ec5205a4d1a already in progress: creating tablet
I20260505 07:53:54.449609 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ee2bff58d3fe472696ac66dc4cfe3ad3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.450107 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ee2bff58d3fe472696ac66dc4cfe3ad3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.451695 1175 tablet_replica.cc:333] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:54.451987 1175 raft_consensus.cc:2243] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.452188 1175 raft_consensus.cc:2272] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.452827 1175 ts_tablet_manager.cc:1916] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.454131 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_468330cd933549a587255653023d8e49" } new_table_name: "default.Table_a006de2dda30423f83e51da1e0477f9d" modify_external_catalogs: true
I20260505 07:53:54.455775 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_e23a9873d96a4ca8ba1bbe8c4976bc59"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.456316 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e23a9873d96a4ca8ba1bbe8c4976bc59 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.458359 900 tablet_service.cc:1558] Processing DeleteTablet for tablet 8573f8bd424641438e2aa6ff6869180e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.458636 893 tablet_service.cc:1558] Processing DeleteTablet for tablet 8713b273c7b8438d8749f595ee5f1629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.458808 1171 raft_consensus.cc:2243] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.459151 1171 raft_consensus.cc:2272] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.459438 899 tablet_service.cc:1558] Processing DeleteTablet for tablet faf1140fe05e4f9493563ab8464c582b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.470808 895 tablet_service.cc:1511] Processing CreateTablet for tablet 32c008bd99224a7595cecc607e5496a9 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.471714 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32c008bd99224a7595cecc607e5496a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.473213 1175 ts_tablet_manager.cc:1929] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.473325 1175 log.cc:1199] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/f296602a34634c649d2573cb178b7053
I20260505 07:53:54.473691 1175 ts_tablet_manager.cc:1950] T f296602a34634c649d2573cb178b7053 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:54.475172 1175 tablet_replica.cc:333] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:54.475379 1175 raft_consensus.cc:2243] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:53:54.474952 901 tablet_service.cc:1511] Processing CreateTablet for tablet 3ac06bb9d273457ead843f5dd754539c (DEFAULT_TABLE table=default.Table_c31ef73b05d44d77b065f8f92c390efb [id=4c5b3219388f43d4866a6ef3b32bf603]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.475991 1175 raft_consensus.cc:2272] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.476023 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet f296602a34634c649d2573cb178b7053 (table default.Table_b3f0f5990c0946598233d0bd2fe0084d [id=db183a96c4214053beda00a79bdd6b44]) successfully deleted
W20260505 07:53:54.476285 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 5b124eda66364b7a81426934b98bca48 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5b124eda66364b7a81426934b98bca48 already in progress: creating tablet
W20260505 07:53:54.476557 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 2850a301ac4f435a96742b00bf54baa9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2850a301ac4f435a96742b00bf54baa9 already in progress: creating tablet
I20260505 07:53:54.477573 1172 tablet_bootstrap.cc:654] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.479620 1175 ts_tablet_manager.cc:1916] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.481664 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 6429919aba17447dbdb393176360bbb0 (DEFAULT_TABLE table=default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.481907 1029 tablet_service.cc:1511] Processing CreateTablet for tablet 571257bbb7e94315b60d84d6aa4917b1 (DEFAULT_TABLE table=default.Table_464b120ca20e45fab0404529a41c17ec [id=6cb8bc8043474bad9f4278b7eef7c71a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.482084 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6429919aba17447dbdb393176360bbb0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:54.482191 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 already in progress: creating tablet
I20260505 07:53:54.482249 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 571257bbb7e94315b60d84d6aa4917b1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:54.482375 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 222c41e6cd8b4d0093fd692618efb107 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 222c41e6cd8b4d0093fd692618efb107 already in progress: creating tablet
I20260505 07:53:54.482697 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 6a2b859a5dd54bc8b36f3197b58afcea (table default.Table_31dd1382f8b4494e9e13ac5b3ac18aa0 [id=b8e0010041cc477d9ce226cca5a28a52]) successfully deleted
I20260505 07:53:54.482663 1027 tablet_service.cc:1511] Processing CreateTablet for tablet b404e50ae092419c9396772b3b14eeaa (DEFAULT_TABLE table=default.Table_a951568d1c5845d9a5329bd1654b99db [id=8f533b7aa2374c439f7234a373af31bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.482988 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b404e50ae092419c9396772b3b14eeaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.483726 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ac06bb9d273457ead843f5dd754539c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.483956 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet 8713b273c7b8438d8749f595ee5f1629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.485234 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet faf1140fe05e4f9493563ab8464c582b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.487732 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 8573f8bd424641438e2aa6ff6869180e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.496232 1220 raft_consensus.cc:493] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:54.496405 1220 raft_consensus.cc:515] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.497109 1220 leader_election.cc:290] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:54.498063 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cce14283e7405690a15e26be8a45b3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:54.498543 717 leader_election.cc:343] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.501116 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cce14283e7405690a15e26be8a45b3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:53:54.506494 1171 ts_tablet_manager.cc:1916] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.519735 1172 tablet_bootstrap.cc:492] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.519734 1178 ts_tablet_manager.cc:1916] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.519884 1172 ts_tablet_manager.cc:1403] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.205s user 0.003s sys 0.001s
I20260505 07:53:54.521497 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 32c008bd99224a7595cecc607e5496a9 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.525277 1178 ts_tablet_manager.cc:1929] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.526810 1178 log.cc:1199] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/f296602a34634c649d2573cb178b7053
I20260505 07:53:54.526907 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32c008bd99224a7595cecc607e5496a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.527258 1178 ts_tablet_manager.cc:1950] T f296602a34634c649d2573cb178b7053 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:54.528641 1172 raft_consensus.cc:359] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.528839 1172 raft_consensus.cc:385] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.529345 1172 raft_consensus.cc:740] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.529613 1172 consensus_queue.cc:260] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.530625 1172 ts_tablet_manager.cc:1434] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260505 07:53:54.530932 1172 tablet_bootstrap.cc:492] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.535439 1175 ts_tablet_manager.cc:1929] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:53:54.536019 1175 log.cc:1199] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/56a783ccd69d453ea7e7c004bf55170e
I20260505 07:53:54.536190 1172 tablet_bootstrap.cc:654] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.536520 1175 ts_tablet_manager.cc:1950] T 56a783ccd69d453ea7e7c004bf55170e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:54.537707 1175 tablet_replica.cc:333] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:54.537964 1175 raft_consensus.cc:2243] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.538165 1175 raft_consensus.cc:2272] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.538816 1175 ts_tablet_manager.cc:1916] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:54.539717 718 leader_election.cc:343] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.540482 718 leader_election.cc:304] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:54.541486 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 56a783ccd69d453ea7e7c004bf55170e (table default.Table_efa325aff89944cc9d011916a21d132e [id=dda64712b2cd4f1286abaf5c08022180]) successfully deleted
I20260505 07:53:54.541937 1220 raft_consensus.cc:2749] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:54.548401 1171 ts_tablet_manager.cc:1929] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:53:54.548533 1171 log.cc:1199] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/56a783ccd69d453ea7e7c004bf55170e
I20260505 07:53:54.549043 1171 ts_tablet_manager.cc:1950] T 56a783ccd69d453ea7e7c004bf55170e P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.550007 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 56a783ccd69d453ea7e7c004bf55170e (table default.Table_efa325aff89944cc9d011916a21d132e [id=dda64712b2cd4f1286abaf5c08022180]) successfully deleted
W20260505 07:53:54.550635 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 already in progress: creating tablet
W20260505 07:53:54.551158 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 222c41e6cd8b4d0093fd692618efb107 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 222c41e6cd8b4d0093fd692618efb107 already in progress: creating tablet
I20260505 07:53:54.551441 1171 tablet_replica.cc:333] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.551723 1171 raft_consensus.cc:2243] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.551929 1171 raft_consensus.cc:2272] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.552665 1171 ts_tablet_manager.cc:1916] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.556383 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_2f6f14275b8847be96f68a761156a854"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.557096 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2f6f14275b8847be96f68a761156a854 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.558295 1171 ts_tablet_manager.cc:1929] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.561098 1171 log.cc:1199] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/8087cab2da594607a0c3f7cf5839c6f3
I20260505 07:53:54.561636 1171 ts_tablet_manager.cc:1950] T 8087cab2da594607a0c3f7cf5839c6f3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.562942 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 8087cab2da594607a0c3f7cf5839c6f3 (table default.Table_53ef916ef6d94510862d6efec4f98487 [id=2b0d644a9b4a490685837b5e427cffb8]) successfully deleted
W20260505 07:53:54.563990 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet b7cce14283e7405690a15e26be8a45b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7cce14283e7405690a15e26be8a45b3 already in progress: creating tablet
I20260505 07:53:54.564499 1229 raft_consensus.cc:493] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:54.564633 1229 raft_consensus.cc:515] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.565279 1229 leader_election.cc:290] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:53:54.566524 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet c7652191b1d44ba59d856a2b0d7017b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7652191b1d44ba59d856a2b0d7017b1 already in progress: creating tablet
W20260505 07:53:54.566855 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 8573f8bd424641438e2aa6ff6869180e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8573f8bd424641438e2aa6ff6869180e already in progress: creating tablet
W20260505 07:53:54.567049 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 8713b273c7b8438d8749f595ee5f1629 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8713b273c7b8438d8749f595ee5f1629 already in progress: creating tablet
W20260505 07:53:54.567231 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet faf1140fe05e4f9493563ab8464c582b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet faf1140fe05e4f9493563ab8464c582b already in progress: creating tablet
I20260505 07:53:54.573256 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a25715f0fa504e92b54c48712c2c2f48" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:54.573551 782 raft_consensus.cc:2468] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:53:54.574160 854 leader_election.cc:304] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:53:54.574507 1229 raft_consensus.cc:2804] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:54.574643 1229 raft_consensus.cc:493] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:54.574731 1229 raft_consensus.cc:3060] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:54.577389 1229 raft_consensus.cc:515] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.578590 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a25715f0fa504e92b54c48712c2c2f48" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:53:54.578764 782 raft_consensus.cc:3060] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:54.580122 1229 leader_election.cc:290] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:54.582283 782 raft_consensus.cc:2468] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:53:54.582708 854 leader_election.cc:304] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:53:54.584242 1229 raft_consensus.cc:2804] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:54.584474 1229 raft_consensus.cc:697] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:53:54.585224 1229 consensus_queue.cc:237] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:54.592247 1172 tablet_bootstrap.cc:492] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.592458 1172 ts_tablet_manager.cc:1403] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.000s
I20260505 07:53:54.593216 1172 raft_consensus.cc:359] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.593483 1172 raft_consensus.cc:385] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.593597 1172 raft_consensus.cc:740] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.593904 1172 consensus_queue.cc:260] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.594390 1172 ts_tablet_manager.cc:1434] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:53:54.595192 1172 tablet_bootstrap.cc:492] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.597620 1172 tablet_bootstrap.cc:654] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.605085 1172 tablet_bootstrap.cc:492] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.605273 1172 ts_tablet_manager.cc:1403] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.001s
I20260505 07:53:54.606038 1172 raft_consensus.cc:359] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.606925 1172 raft_consensus.cc:385] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.607160 1172 raft_consensus.cc:740] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.607762 1172 consensus_queue.cc:260] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.609670 1172 ts_tablet_manager.cc:1434] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260505 07:53:54.610906 1172 tablet_bootstrap.cc:492] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.612846 1172 tablet_bootstrap.cc:654] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.622212 1172 tablet_bootstrap.cc:492] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.622778 1172 ts_tablet_manager.cc:1403] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
W20260505 07:53:54.622836 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3ac06bb9d273457ead843f5dd754539c consensus state has no leader
I20260505 07:53:54.624281 1172 raft_consensus.cc:359] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.624501 1172 raft_consensus.cc:385] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.624567 1172 raft_consensus.cc:740] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.625213 1172 consensus_queue.cc:260] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.627918 1172 ts_tablet_manager.cc:1434] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260505 07:53:54.629149 1172 tablet_bootstrap.cc:492] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.629720 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_eae91d411d9149658200d44f4837f486" } modify_external_catalogs: true
I20260505 07:53:54.629931 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_eae91d411d9149658200d44f4837f486" } modify_external_catalogs: true
I20260505 07:53:54.630939 1172 tablet_bootstrap.cc:654] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.631287 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_476efdff9cee417b941d0796953265f5" } new_table_name: "default.Table_024c44160bb5470399f3fe342ac501bb" modify_external_catalogs: true
I20260505 07:53:54.633261 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 56a783ccd69d453ea7e7c004bf55170e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:54.633872 588 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 56a783ccd69d453ea7e7c004bf55170e because the tablet was not found. No further retry: Not found: Tablet not found: 56a783ccd69d453ea7e7c004bf55170e
I20260505 07:53:54.634502 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 451e99a0dfec4823812e5991147b132e from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:54.635190 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 62d4d96bdf674c2395938b9971c0ad28 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:54.635749 601 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_70e3149cee264c889c7677c3757ef266" } modify_external_catalogs: true
I20260505 07:53:54.636040 601 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_70e3149cee264c889c7677c3757ef266" } modify_external_catalogs: true
W20260505 07:53:54.636760 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b404e50ae092419c9396772b3b14eeaa consensus state has no leader
I20260505 07:53:54.637109 600 master_service.cc:954] ReplaceTablet: received request to replace tablet 5b124eda66364b7a81426934b98bca48 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:54.637818 600 master_service.cc:954] ReplaceTablet: received request to replace tablet 5b124eda66364b7a81426934b98bca48 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:54.638466 600 master_service.cc:954] ReplaceTablet: received request to replace tablet 62d4d96bdf674c2395938b9971c0ad28 from {username='slave'} at 127.0.0.1:47238
W20260505 07:53:54.640064 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1765a95cc3414c73b7e3918913a35937 consensus state has no leader
I20260505 07:53:54.640508 1229 raft_consensus.cc:493] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:54.640658 1229 raft_consensus.cc:515] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.641163 1229 leader_election.cc:290] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:54.643069 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb76b7961844443a9b41353c2d77d5a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:54.643877 854 leader_election.cc:343] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.647037 1172 tablet_bootstrap.cc:492] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.647553 1172 ts_tablet_manager.cc:1403] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20260505 07:53:54.648228 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7af69942bd9b48e79bc11f90b8fd656c" } new_table_name: "default.Table_b8745d36d43a4f42a5b3d43a37c21e55" modify_external_catalogs: true
I20260505 07:53:54.648663 1172 raft_consensus.cc:359] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.648840 1172 raft_consensus.cc:385] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.648934 1172 raft_consensus.cc:740] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.649983 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_a006de2dda30423f83e51da1e0477f9d" } modify_external_catalogs: true
I20260505 07:53:54.650235 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_a006de2dda30423f83e51da1e0477f9d" } modify_external_catalogs: true
I20260505 07:53:54.650425 1172 consensus_queue.cc:260] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.650962 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ae34ec4a8b52453dba7158ce065f4798" } modify_external_catalogs: true
I20260505 07:53:54.651201 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ae34ec4a8b52453dba7158ce065f4798" } modify_external_catalogs: true
I20260505 07:53:54.652133 1172 ts_tablet_manager.cc:1434] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260505 07:53:54.654343 1172 tablet_bootstrap.cc:492] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.655860 1172 tablet_bootstrap.cc:654] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.661962 1172 tablet_bootstrap.cc:492] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.662110 1172 ts_tablet_manager.cc:1403] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.001s
I20260505 07:53:54.662887 1172 raft_consensus.cc:359] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.663064 1172 raft_consensus.cc:385] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.663127 1172 raft_consensus.cc:740] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.663048 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a25715f0fa504e92b54c48712c2c2f48" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:54.665385 1172 consensus_queue.cc:260] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.666626 1172 ts_tablet_manager.cc:1434] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260505 07:53:54.667043 1172 tablet_bootstrap.cc:492] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:53:54.668228 856 leader_election.cc:343] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.669000 1172 tablet_bootstrap.cc:654] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.669003 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a25715f0fa504e92b54c48712c2c2f48" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:53:54.670254 856 leader_election.cc:343] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.672855 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edb76b7961844443a9b41353c2d77d5a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:54.673523 856 leader_election.cc:343] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.673693 856 leader_election.cc:304] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:54.674168 1229 raft_consensus.cc:2749] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:54.674885 1172 tablet_bootstrap.cc:492] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.675014 1172 ts_tablet_manager.cc:1403] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260505 07:53:54.675905 1172 raft_consensus.cc:359] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:54.676126 1172 raft_consensus.cc:385] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.676196 1172 raft_consensus.cc:740] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.678186 1172 consensus_queue.cc:260] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:54.679126 1172 ts_tablet_manager.cc:1434] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:53:54.679394 1172 tablet_bootstrap.cc:492] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.681828 1172 tablet_bootstrap.cc:654] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.683491 1229 raft_consensus.cc:493] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:54.683689 1229 raft_consensus.cc:515] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.684417 1229 leader_election.cc:290] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:54.684947 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772a65f4016843d18fbca25842f76de0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:54.685164 782 raft_consensus.cc:2468] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:53:54.685175 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772a65f4016843d18fbca25842f76de0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:54.685572 854 leader_election.cc:304] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
W20260505 07:53:54.685833 856 leader_election.cc:343] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.685891 1229 raft_consensus.cc:2804] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:54.686245 1229 raft_consensus.cc:493] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:54.686386 1229 raft_consensus.cc:3060] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:54.689494 1172 tablet_bootstrap.cc:492] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.689641 1172 ts_tablet_manager.cc:1403] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260505 07:53:54.690366 1172 raft_consensus.cc:359] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.690577 1172 raft_consensus.cc:385] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.690652 1172 raft_consensus.cc:740] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.690678 1229 raft_consensus.cc:515] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.692092 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772a65f4016843d18fbca25842f76de0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:53:54.692322 782 raft_consensus.cc:3060] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:54.692487 1172 consensus_queue.cc:260] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.694867 1172 ts_tablet_manager.cc:1434] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260505 07:53:54.695147 1172 tablet_bootstrap.cc:492] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.696386 782 raft_consensus.cc:2468] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:53:54.696969 854 leader_election.cc:304] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:53:54.697598 1180 raft_consensus.cc:2804] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:54.697773 1180 raft_consensus.cc:697] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:53:54.698091 1180 consensus_queue.cc:237] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.698266 1172 tablet_bootstrap.cc:654] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.700512 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "772a65f4016843d18fbca25842f76de0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:53:54.701429 856 leader_election.cc:343] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.702057 1229 leader_election.cc:290] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:54.703871 1172 tablet_bootstrap.cc:492] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.703989 1172 ts_tablet_manager.cc:1403] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260505 07:53:54.704660 1172 raft_consensus.cc:359] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.704828 1172 raft_consensus.cc:385] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.704910 1172 raft_consensus.cc:740] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.706530 1172 consensus_queue.cc:260] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.708101 1172 ts_tablet_manager.cc:1434] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:53:54.709306 1172 tablet_bootstrap.cc:492] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.711210 1172 tablet_bootstrap.cc:654] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.714576 901 tablet_service.cc:1558] Processing DeleteTablet for tablet edb76b7961844443a9b41353c2d77d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.714860 1171 tablet_replica.cc:333] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.715021 1171 raft_consensus.cc:2243] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.715193 1171 raft_consensus.cc:2272] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.715900 1171 ts_tablet_manager.cc:1916] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.720641 1172 tablet_bootstrap.cc:492] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.720813 1172 ts_tablet_manager.cc:1403] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.004s
I20260505 07:53:54.721539 1172 raft_consensus.cc:359] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.721717 1172 raft_consensus.cc:385] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.721796 1172 raft_consensus.cc:740] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.722357 1171 ts_tablet_manager.cc:1929] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.722508 1171 log.cc:1199] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/edb76b7961844443a9b41353c2d77d5a
I20260505 07:53:54.723008 1171 ts_tablet_manager.cc:1950] T edb76b7961844443a9b41353c2d77d5a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.723281 1172 consensus_queue.cc:260] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.725188 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 05b8c0a2185e473cbb4165fed9e94c8a with delete_type TABLET_DATA_DELETED (replaced by tablet 5f57568e1c4c4573ac18d9f621eeb4c7) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.725657 1171 tablet_replica.cc:333] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.725679 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet edb76b7961844443a9b41353c2d77d5a (table default.Table_dda9e1fe74264c2eab25cecc65079c93 [id=50e792c4b9044685ab0605ef0a9474a5]) successfully deleted
I20260505 07:53:54.725942 1171 raft_consensus.cc:2243] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.726549 1171 raft_consensus.cc:2272] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.727080 1172 ts_tablet_manager.cc:1434] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260505 07:53:54.728569 1172 tablet_bootstrap.cc:492] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.728708 1171 ts_tablet_manager.cc:1916] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.730250 1172 tablet_bootstrap.cc:654] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.730906 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 772a65f4016843d18fbca25842f76de0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.732852 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 8713b273c7b8438d8749f595ee5f1629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.738512 1220 raft_consensus.cc:493] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:54.738751 1220 raft_consensus.cc:515] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.739609 1220 leader_election.cc:290] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:54.741183 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b8c0a2185e473cbb4165fed9e94c8a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:53:54.742120 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05b8c0a2185e473cbb4165fed9e94c8a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:54.742759 717 leader_election.cc:343] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:53:54.743119 718 leader_election.cc:343] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:53:54.743243 718 leader_election.cc:304] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:54.743805 1220 raft_consensus.cc:2749] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:54.743970 1171 ts_tablet_manager.cc:1929] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.744112 1171 log.cc:1199] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/05b8c0a2185e473cbb4165fed9e94c8a
I20260505 07:53:54.744549 1171 ts_tablet_manager.cc:1950] T 05b8c0a2185e473cbb4165fed9e94c8a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
W20260505 07:53:54.745901 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1a9420a81844d2092381123d9297661 consensus state has no leader
I20260505 07:53:54.748565 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4e9260f6bf4c4ea3b35c44a98dcf15f1" } modify_external_catalogs: true
I20260505 07:53:54.748780 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4e9260f6bf4c4ea3b35c44a98dcf15f1" } modify_external_catalogs: true
I20260505 07:53:54.753988 901 tablet_service.cc:1511] Processing CreateTablet for tablet 4a79578757824bfc9df129e3475e7af3 (DEFAULT_TABLE table=default.Table_b31d40ee188f46d0bdd481aa88730544 [id=c6653c0d02c64683870dc0e9551fcaf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.754392 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a79578757824bfc9df129e3475e7af3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.755861 1171 tablet_replica.cc:333] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.756052 1171 raft_consensus.cc:2243] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:53:54.756302 1171 pending_rounds.cc:70] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:53:54.756381 1171 pending_rounds.cc:77] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555431726174208 op_type: NO_OP noop_request { }
I20260505 07:53:54.756507 1171 raft_consensus.cc:2889] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:53:54.756608 1171 raft_consensus.cc:2272] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.757205 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 05b8c0a2185e473cbb4165fed9e94c8a (table default.Table_1510db6bb88343d0aa4e95a01ecf17a6 [id=ad11a4d7ee734c9bab8278aebac5cb40]) successfully deleted
I20260505 07:53:54.757380 1171 ts_tablet_manager.cc:1916] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.757342 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 370be7a513ee49028c8d0662b43c8d91 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dab02a546b944f799045723a515118f) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.764288 1171 ts_tablet_manager.cc:1929] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:53:54.764457 1171 log.cc:1199] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/772a65f4016843d18fbca25842f76de0
I20260505 07:53:54.764935 1171 ts_tablet_manager.cc:1950] T 772a65f4016843d18fbca25842f76de0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.765899 1171 tablet_replica.cc:333] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.765957 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 772a65f4016843d18fbca25842f76de0 (table default.Table_4805d11726a3446b86b7a990c0dbb3c5 [id=035a76f8e6f34bc1bb1c75c4908bb4d7]) successfully deleted
I20260505 07:53:54.766119 1171 raft_consensus.cc:2243] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.766336 1171 raft_consensus.cc:2272] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.770288 901 tablet_service.cc:1558] Processing DeleteTablet for tablet a25715f0fa504e92b54c48712c2c2f48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.770767 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_c61fca437ddb413e934595a23b52f40f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.776125 1171 ts_tablet_manager.cc:1916] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.783785 1171 ts_tablet_manager.cc:1929] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.783921 1171 log.cc:1199] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/8713b273c7b8438d8749f595ee5f1629
I20260505 07:53:54.784456 1171 ts_tablet_manager.cc:1950] T 8713b273c7b8438d8749f595ee5f1629 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.785468 1171 tablet_replica.cc:333] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.785737 1171 raft_consensus.cc:2243] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.785923 1171 raft_consensus.cc:2272] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.790040 1171 ts_tablet_manager.cc:1916] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.797057 1171 ts_tablet_manager.cc:1929] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.797207 1171 log.cc:1199] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/370be7a513ee49028c8d0662b43c8d91
I20260505 07:53:54.797607 1171 ts_tablet_manager.cc:1950] T 370be7a513ee49028c8d0662b43c8d91 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.797653 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 4a79578757824bfc9df129e3475e7af3 (DEFAULT_TABLE table=default.Table_b31d40ee188f46d0bdd481aa88730544 [id=c6653c0d02c64683870dc0e9551fcaf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.798120 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a79578757824bfc9df129e3475e7af3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.807832 1171 tablet_replica.cc:333] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.808091 1171 raft_consensus.cc:2243] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:53:54.808405 1171 pending_rounds.cc:70] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:53:54.808465 1171 pending_rounds.cc:77] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555431265136640 op_type: NO_OP noop_request { }
I20260505 07:53:54.808598 1171 raft_consensus.cc:2889] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:53:54.808715 1171 raft_consensus.cc:2272] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.820286 1171 ts_tablet_manager.cc:1916] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.828012 1171 ts_tablet_manager.cc:1929] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260505 07:53:54.771737 585 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 05b8c0a2185e473cbb4165fed9e94c8a (table default.Table_1510db6bb88343d0aa4e95a01ecf17a6 [id=ad11a4d7ee734c9bab8278aebac5cb40], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 05b8c0a2185e473cbb4165fed9e94c8a consensus state has no leader
I20260505 07:53:54.828181 1171 log.cc:1199] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/a25715f0fa504e92b54c48712c2c2f48
I20260505 07:53:54.787963 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 8713b273c7b8438d8749f595ee5f1629 (table default.Table_aa53055d0748464697a41feba3c4f036 [id=af8d003d14d24f34bd57e804562d1f33]) successfully deleted
I20260505 07:53:54.828979 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 370be7a513ee49028c8d0662b43c8d91 (table default.Table_448efd98249942379556e125b25d169c [id=b2adaf7422c7403c8b331c0d33f0ad53]) successfully deleted
I20260505 07:53:54.829144 901 tablet_service.cc:1511] Processing CreateTablet for tablet 9959be9f83c2444282d012b61eef49b5 (DEFAULT_TABLE table=default.Table_ae34ec4a8b52453dba7158ce065f4798 [id=8314d2bc7c1a401db1d6fac29fff5c3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.829316 1171 ts_tablet_manager.cc:1950] T a25715f0fa504e92b54c48712c2c2f48 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.829581 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9959be9f83c2444282d012b61eef49b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.830420 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet a25715f0fa504e92b54c48712c2c2f48 (table default.Table_23d3119ff64f4b9caf3ce96d7520fccf [id=0ac6c7c427534982aa4dca340c2001bc]) successfully deleted
I20260505 07:53:54.831993 761 tablet_service.cc:1511] Processing CreateTablet for tablet 03f31efbc98c484c950c70d5e84d2136 (DEFAULT_TABLE table=default.Table_6ff000247d324fe8b55b1398e48cb49a [id=8903351259c347a1925769663d34d173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.832748 755 tablet_service.cc:1511] Processing CreateTablet for tablet 9959be9f83c2444282d012b61eef49b5 (DEFAULT_TABLE table=default.Table_ae34ec4a8b52453dba7158ce065f4798 [id=8314d2bc7c1a401db1d6fac29fff5c3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.833432 750 tablet_service.cc:1511] Processing CreateTablet for tablet 4a79578757824bfc9df129e3475e7af3 (DEFAULT_TABLE table=default.Table_b31d40ee188f46d0bdd481aa88730544 [id=c6653c0d02c64683870dc0e9551fcaf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.798081 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet f296602a34634c649d2573cb178b7053 (table default.Table_b3f0f5990c0946598233d0bd2fe0084d [id=db183a96c4214053beda00a79bdd6b44]) successfully deleted
W20260505 07:53:54.804502 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e85b44337f62494c997c0008fc492aab consensus state has no leader
I20260505 07:53:54.811861 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 09a8dbb7fe4b406d804c0b62ba006adb
I20260505 07:53:54.829213 895 tablet_service.cc:1511] Processing CreateTablet for tablet 03f31efbc98c484c950c70d5e84d2136 (DEFAULT_TABLE table=default.Table_6ff000247d324fe8b55b1398e48cb49a [id=8903351259c347a1925769663d34d173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.835933 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03f31efbc98c484c950c70d5e84d2136. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:54.836350 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet c7652191b1d44ba59d856a2b0d7017b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7652191b1d44ba59d856a2b0d7017b1 already in progress: creating tablet
I20260505 07:53:54.820439 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 1765a95cc3414c73b7e3918913a35937
W20260505 07:53:54.837951 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet b7cce14283e7405690a15e26be8a45b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b7cce14283e7405690a15e26be8a45b3 already in progress: creating tablet
W20260505 07:53:54.838969 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8713b273c7b8438d8749f595ee5f1629 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8713b273c7b8438d8749f595ee5f1629 already in progress: creating tablet
W20260505 07:53:54.839278 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet faf1140fe05e4f9493563ab8464c582b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet faf1140fe05e4f9493563ab8464c582b already in progress: creating tablet
W20260505 07:53:54.839617 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8573f8bd424641438e2aa6ff6869180e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8573f8bd424641438e2aa6ff6869180e already in progress: creating tablet
I20260505 07:53:54.839740 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 09a8dbb7fe4b406d804c0b62ba006adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.840011 1171 tablet_replica.cc:333] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.840211 1171 raft_consensus.cc:2243] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.840411 1171 raft_consensus.cc:2272] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.840602 897 tablet_service.cc:1558] Processing DeleteTablet for tablet 1765a95cc3414c73b7e3918913a35937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.841063 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 9959be9f83c2444282d012b61eef49b5 (DEFAULT_TABLE table=default.Table_ae34ec4a8b52453dba7158ce065f4798 [id=8314d2bc7c1a401db1d6fac29fff5c3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.841171 1171 ts_tablet_manager.cc:1916] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.841480 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9959be9f83c2444282d012b61eef49b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.841572 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 03f31efbc98c484c950c70d5e84d2136 (DEFAULT_TABLE table=default.Table_6ff000247d324fe8b55b1398e48cb49a [id=8903351259c347a1925769663d34d173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:54.841871 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03f31efbc98c484c950c70d5e84d2136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.849628 1171 ts_tablet_manager.cc:1929] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.849757 1171 log.cc:1199] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/09a8dbb7fe4b406d804c0b62ba006adb
I20260505 07:53:54.850162 1171 ts_tablet_manager.cc:1950] T 09a8dbb7fe4b406d804c0b62ba006adb P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.859867 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 09a8dbb7fe4b406d804c0b62ba006adb (table default.Table_eae91d411d9149658200d44f4837f486 [id=789cc9783535475e9929332f36141e27]) successfully deleted
W20260505 07:53:54.860224 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 1765a95cc3414c73b7e3918913a35937 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1765a95cc3414c73b7e3918913a35937 already in progress: creating tablet
I20260505 07:53:54.862749 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 09a8dbb7fe4b406d804c0b62ba006adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.865809 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 1765a95cc3414c73b7e3918913a35937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:54.866992 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 09a8dbb7fe4b406d804c0b62ba006adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.867604 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 1765a95cc3414c73b7e3918913a35937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.867997 1175 ts_tablet_manager.cc:1929] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.868122 1175 log.cc:1199] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/0fd990e80a90418dbb9b94cb6e2ee156
I20260505 07:53:54.868602 1175 ts_tablet_manager.cc:1950] T 0fd990e80a90418dbb9b94cb6e2ee156 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:54.869012 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03f31efbc98c484c950c70d5e84d2136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.869691 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9959be9f83c2444282d012b61eef49b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:54.870605 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a79578757824bfc9df129e3475e7af3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:54.883684 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c61fca437ddb413e934595a23b52f40f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.896754 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 0fd990e80a90418dbb9b94cb6e2ee156 (table default.Table_0eea08e8b83741208451d0a496b53a5c [id=40e8397fa22f44ecb51f34f6f2ac2408]) successfully deleted
I20260505 07:53:54.913823 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_3c9a0977f3244728b0e4de172b1559d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.914883 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c9a0977f3244728b0e4de172b1559d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.918591 595 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_31dd1382f8b4494e9e13ac5b3ac18aa0 [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 3 got 0
I20260505 07:53:54.920886 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 2850a301ac4f435a96742b00bf54baa9 with delete_type TABLET_DATA_DELETED (replaced by tablet 956b5e05bda648349ba26d5c1ea3b279) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.921195 1171 tablet_replica.cc:333] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.921450 1171 raft_consensus.cc:2243] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.921651 1171 raft_consensus.cc:2272] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.921705 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_e0534e758791440aa17726afbfbf40aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:54.922549 1171 ts_tablet_manager.cc:1916] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:54.922556 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e0534e758791440aa17726afbfbf40aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.925369 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 09a8dbb7fe4b406d804c0b62ba006adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.925904 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.925909 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b124eda66364b7a81426934b98bca48 with delete_type TABLET_DATA_DELETED (replaced by tablet 3781d0bbcd1b4189bd836eeae4236f40) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.926337 895 tablet_service.cc:1558] Processing DeleteTablet for tablet b7cce14283e7405690a15e26be8a45b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 6429919aba17447dbdb393176360bbb0) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.926757 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_3a027f114e1e4d5fabe7ddc8f584e3ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.927520 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3a027f114e1e4d5fabe7ddc8f584e3ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.929906 1171 ts_tablet_manager.cc:1929] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.930040 1171 log.cc:1199] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/2850a301ac4f435a96742b00bf54baa9
I20260505 07:53:54.930410 1171 ts_tablet_manager.cc:1950] T 2850a301ac4f435a96742b00bf54baa9 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.932221 1171 tablet_replica.cc:333] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.932437 1171 raft_consensus.cc:2243] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.932395 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 2850a301ac4f435a96742b00bf54baa9 (table default.Table_74bdaa65d97949c9ae16b85c62dbcc59 [id=d524458d873a48cb980e2f85490232d6]) successfully deleted
I20260505 07:53:54.932626 1171 raft_consensus.cc:2272] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:53:54.932894 588 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 09a8dbb7fe4b406d804c0b62ba006adb because the tablet was not found. No further retry: Not found: Tablet not found: 09a8dbb7fe4b406d804c0b62ba006adb
I20260505 07:53:54.933396 1171 ts_tablet_manager.cc:1916] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.943780 601 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 3272555a8c814c0ebe61ca32d715212b
I20260505 07:53:54.945489 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 3272555a8c814c0ebe61ca32d715212b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.946655 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_786d64e187254c7880f44b3d95290cc5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:54.947316 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_786d64e187254c7880f44b3d95290cc5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:54.953773 1171 ts_tablet_manager.cc:1929] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.953900 1171 log.cc:1199] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/31b151ff45804fbaa76cfbec86c6fd7a
I20260505 07:53:54.954327 1171 ts_tablet_manager.cc:1950] T 31b151ff45804fbaa76cfbec86c6fd7a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.955276 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 31b151ff45804fbaa76cfbec86c6fd7a (table default.Table_7f23cbfec9794a2bbc9e9dad2d07011c [id=b25c09d8fb8347af83c69e76a523867e]) successfully deleted
I20260505 07:53:54.955812 1172 tablet_bootstrap.cc:492] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.955966 1172 ts_tablet_manager.cc:1403] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.228s user 0.030s sys 0.053s
W20260505 07:53:54.955681 587 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.61:33565, user_credentials={real_user=slave}} blocked reactor thread for 70229.2us
W20260505 07:53:54.956526 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 09a8dbb7fe4b406d804c0b62ba006adb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09a8dbb7fe4b406d804c0b62ba006adb already in progress: creating tablet
I20260505 07:53:54.956745 1172 raft_consensus.cc:359] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.956945 1172 raft_consensus.cc:385] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.957042 1172 raft_consensus.cc:740] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:53:54.957273 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 1765a95cc3414c73b7e3918913a35937 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1765a95cc3414c73b7e3918913a35937 already in progress: creating tablet
I20260505 07:53:54.958920 1171 tablet_replica.cc:333] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.959141 1171 raft_consensus.cc:2243] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.959297 1171 raft_consensus.cc:2272] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.959745 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 3272555a8c814c0ebe61ca32d715212b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.959813 1172 consensus_queue.cc:260] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:54.960650 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 3272555a8c814c0ebe61ca32d715212b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3272555a8c814c0ebe61ca32d715212b already in progress: creating tablet
I20260505 07:53:54.962111 1172 ts_tablet_manager.cc:1434] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:53:54.962652 1229 raft_consensus.cc:493] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:54.962805 1229 raft_consensus.cc:515] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.963406 1229 leader_election.cc:290] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:54.964349 1171 ts_tablet_manager.cc:1916] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:54.965843 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca4cf248adb4f768c5586d72614ad42" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:54.966080 1172 tablet_bootstrap.cc:492] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:53:54.966768 856 leader_election.cc:343] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:54.968924 1172 tablet_bootstrap.cc:654] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.974517 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 451e99a0dfec4823812e5991147b132e
I20260505 07:53:54.975163 594 catalog_manager.cc:6741] ReplaceTablet: tablet 451e99a0dfec4823812e5991147b132e deleted and replaced by tablet fcd36b5a84b4442fb76e1610d8af4c07
I20260505 07:53:54.977135 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 451e99a0dfec4823812e5991147b132e with delete_type TABLET_DATA_DELETED (replaced by tablet fcd36b5a84b4442fb76e1610d8af4c07) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:54.979341 1172 tablet_bootstrap.cc:492] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:54.979640 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 451e99a0dfec4823812e5991147b132e with delete_type TABLET_DATA_DELETED (replaced by tablet fcd36b5a84b4442fb76e1610d8af4c07) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:54.979964 1172 ts_tablet_manager.cc:1403] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
W20260505 07:53:54.980615 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 451e99a0dfec4823812e5991147b132e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 451e99a0dfec4823812e5991147b132e already in progress: creating tablet
I20260505 07:53:54.982018 1172 raft_consensus.cc:359] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.982093 594 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4bd82b12eff3408eaa6b8b4d7413e3f5 [id=b86b2850fe6f49b1907835ae468cf22e]). Expected version 1 got 0
I20260505 07:53:54.982208 1172 raft_consensus.cc:385] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:54.982267 1172 raft_consensus.cc:740] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:54.983177 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ee2bff58d3fe472696ac66dc4cfe3ad3" } new_table_name: "default.Table_d9857ceb9b4d48dab8120e696f98e0e5" modify_external_catalogs: true
I20260505 07:53:54.984215 1172 consensus_queue.cc:260] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:54.986604 1172 ts_tablet_manager.cc:1434] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:53:54.988042 1172 tablet_bootstrap.cc:492] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:54.990192 1172 tablet_bootstrap.cc:654] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:54.991727 1171 ts_tablet_manager.cc:1929] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:54.991852 1171 log.cc:1199] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/5b124eda66364b7a81426934b98bca48
I20260505 07:53:54.992179 1171 ts_tablet_manager.cc:1950] T 5b124eda66364b7a81426934b98bca48 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:54.993321 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 5b124eda66364b7a81426934b98bca48 (table default.Table_8e821d63b81c43258a16db52d5295ad2 [id=b4934f65dfeb4a679111783301bb1fb9]) successfully deleted
I20260505 07:53:54.993865 1220 raft_consensus.cc:493] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:54.995406 1171 tablet_replica.cc:333] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:54.995999 1171 raft_consensus.cc:2243] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:54.996182 1171 raft_consensus.cc:2272] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:54.997642 1171 ts_tablet_manager.cc:1916] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.003116 1172 tablet_bootstrap.cc:492] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.003255 1172 ts_tablet_manager.cc:1403] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260505 07:53:55.005318 1172 raft_consensus.cc:359] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.005537 1172 raft_consensus.cc:385] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.005640 1172 raft_consensus.cc:740] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.006888 1172 consensus_queue.cc:260] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.009228 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca4cf248adb4f768c5586d72614ad42" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:55.009851 1172 ts_tablet_manager.cc:1434] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
W20260505 07:53:55.010018 854 leader_election.cc:343] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.010205 854 leader_election.cc:304] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:55.010723 1229 raft_consensus.cc:2749] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:55.011390 1172 tablet_bootstrap.cc:492] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.013935 1172 tablet_bootstrap.cc:654] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.015874 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 3272555a8c814c0ebe61ca32d715212b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.016525 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 451e99a0dfec4823812e5991147b132e with delete_type TABLET_DATA_DELETED (replaced by tablet fcd36b5a84b4442fb76e1610d8af4c07) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.017377 1220 raft_consensus.cc:515] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.017688 1171 ts_tablet_manager.cc:1929] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.017805 1171 log.cc:1199] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/b7cce14283e7405690a15e26be8a45b3
I20260505 07:53:55.018209 1175 tablet_replica.cc:333] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.018445 1175 raft_consensus.cc:2243] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
W20260505 07:53:55.018455 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 already in progress: creating tablet
I20260505 07:53:55.018615 1175 raft_consensus.cc:2272] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.018702 1171 ts_tablet_manager.cc:1950] T b7cce14283e7405690a15e26be8a45b3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.019198 1175 ts_tablet_manager.cc:1916] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:55.019577 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 222c41e6cd8b4d0093fd692618efb107 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 222c41e6cd8b4d0093fd692618efb107 already in progress: creating tablet
I20260505 07:53:55.019991 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet b7cce14283e7405690a15e26be8a45b3 (table default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]) successfully deleted
I20260505 07:53:55.022833 1172 tablet_bootstrap.cc:492] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.022985 1172 ts_tablet_manager.cc:1403] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260505 07:53:55.023788 1172 raft_consensus.cc:359] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.024004 1172 raft_consensus.cc:385] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.024076 1172 raft_consensus.cc:740] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.026046 1172 consensus_queue.cc:260] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.026583 1172 ts_tablet_manager.cc:1434] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:53:55.029711 1172 tablet_bootstrap.cc:492] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.030707 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b151ff45804fbaa76cfbec86c6fd7a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:55.031291 1172 tablet_bootstrap.cc:654] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:55.032534 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 3272555a8c814c0ebe61ca32d715212b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3272555a8c814c0ebe61ca32d715212b already in progress: creating tablet
W20260505 07:53:55.032859 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 451e99a0dfec4823812e5991147b132e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 451e99a0dfec4823812e5991147b132e already in progress: creating tablet
I20260505 07:53:55.036722 1172 tablet_bootstrap.cc:492] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.036859 1172 ts_tablet_manager.cc:1403] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260505 07:53:55.037672 1172 raft_consensus.cc:359] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.037832 1172 raft_consensus.cc:385] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.037916 1172 raft_consensus.cc:740] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.039857 1172 consensus_queue.cc:260] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.042263 1172 ts_tablet_manager.cc:1434] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260505 07:53:55.042541 1172 tablet_bootstrap.cc:492] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.045208 1172 tablet_bootstrap.cc:654] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.047086 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 62d4d96bdf674c2395938b9971c0ad28
I20260505 07:53:55.047691 593 catalog_manager.cc:6741] ReplaceTablet: tablet 62d4d96bdf674c2395938b9971c0ad28 deleted and replaced by tablet 7e707143a4e64725aa66db05b24999a6
I20260505 07:53:55.050385 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4d96bdf674c2395938b9971c0ad28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e707143a4e64725aa66db05b24999a6) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:55.051061 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 62d4d96bdf674c2395938b9971c0ad28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62d4d96bdf674c2395938b9971c0ad28 already in progress: creating tablet
I20260505 07:53:55.052011 1172 tablet_bootstrap.cc:492] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.052405 1172 ts_tablet_manager.cc:1403] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260505 07:53:55.053282 1172 raft_consensus.cc:359] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.054272 1172 raft_consensus.cc:385] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.054507 1172 raft_consensus.cc:740] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.054858 1172 consensus_queue.cc:260] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.056578 1172 ts_tablet_manager.cc:1434] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260505 07:53:55.058177 1172 tablet_bootstrap.cc:492] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.058209 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4d96bdf674c2395938b9971c0ad28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e707143a4e64725aa66db05b24999a6) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:55.059382 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 62d4d96bdf674c2395938b9971c0ad28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62d4d96bdf674c2395938b9971c0ad28 already in progress: creating tablet
I20260505 07:53:55.060402 1172 tablet_bootstrap.cc:654] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.066083 1172 tablet_bootstrap.cc:492] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.066255 1172 ts_tablet_manager.cc:1403] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260505 07:53:55.067235 1172 raft_consensus.cc:359] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.067459 1172 raft_consensus.cc:385] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.067548 1172 raft_consensus.cc:740] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.069203 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_66d69d43d5d24b9b88a4d885b0ce8f1d" } new_table_name: "default.Table_efa75c9c9cb34200856f6e2816aa4deb" modify_external_catalogs: true
I20260505 07:53:55.069736 1172 consensus_queue.cc:260] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.070627 1172 ts_tablet_manager.cc:1434] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:53:55.070865 1172 tablet_bootstrap.cc:492] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.072474 593 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4bd82b12eff3408eaa6b8b4d7413e3f5 [id=b86b2850fe6f49b1907835ae468cf22e]). Expected version 1 got 0
I20260505 07:53:55.073391 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_02afe59354994bffb24b190e9a7abf85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.074090 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_02afe59354994bffb24b190e9a7abf85 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.076200 1172 tablet_bootstrap.cc:654] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.083693 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 9959be9f83c2444282d012b61eef49b5
I20260505 07:53:55.084715 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 9959be9f83c2444282d012b61eef49b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.089548 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 9959be9f83c2444282d012b61eef49b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:55.090559 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 9959be9f83c2444282d012b61eef49b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9959be9f83c2444282d012b61eef49b5 already in progress: creating tablet
W20260505 07:53:55.091116 1181 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.864s user 0.113s sys 0.186s
W20260505 07:53:55.091235 1181 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.864s user 0.113s sys 0.186s
I20260505 07:53:55.091974 1220 leader_election.cc:290] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:55.092105 1175 ts_tablet_manager.cc:1929] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.092303 1181 tablet_bootstrap.cc:492] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:55.092520 1181 ts_tablet_manager.cc:1403] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.888s user 0.115s sys 0.186s
I20260505 07:53:55.093293 1181 raft_consensus.cc:359] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.093585 1181 raft_consensus.cc:385] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.093710 1181 raft_consensus.cc:740] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.094019 1181 consensus_queue.cc:260] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.095901 1273 raft_consensus.cc:493] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:55.096747 1273 raft_consensus.cc:515] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.097546 1175 log.cc:1199] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/070d1b1b30d447539911349841638c12
I20260505 07:53:55.097867 1273 leader_election.cc:290] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:55.102162 1175 ts_tablet_manager.cc:1950] T 070d1b1b30d447539911349841638c12 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.103052 1175 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34194 (request call id 61) took 1021 ms. Trace:
I20260505 07:53:55.103178 1175 rpcz_store.cc:276] 0505 07:53:54.081913 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:54.116972 (+ 35059us) service_pool.cc:224] Handling call
0505 07:53:55.001495 (+884523us) ts_tablet_manager.cc:1151] Deleting tablet 070d1b1b30d447539911349841638c12
0505 07:53:55.001507 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:55.022516 (+ 21009us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.025630 (+ 3114us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.102935 (+ 77305us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":882412}
I20260505 07:53:55.104350 1175 tablet_replica.cc:333] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.104581 1175 raft_consensus.cc:2243] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.104935 1175 raft_consensus.cc:2272] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.105583 1175 ts_tablet_manager.cc:1916] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:55.106321 1237 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260505 07:53:55.106702 1237 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260505 07:53:55.106820 1237 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260505 07:53:55.116828 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 3ac06bb9d273457ead843f5dd754539c
I20260505 07:53:55.118392 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ac06bb9d273457ead843f5dd754539c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:55.119704 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 9959be9f83c2444282d012b61eef49b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9959be9f83c2444282d012b61eef49b5 already in progress: creating tablet
W20260505 07:53:55.119992 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 3ac06bb9d273457ead843f5dd754539c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ac06bb9d273457ead843f5dd754539c already in progress: creating tablet
I20260505 07:53:55.121196 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4bd82b12eff3408eaa6b8b4d7413e3f5" } new_table_name: "default.Table_4011b7c55a4e4e149f8aa0b6f2981824" modify_external_catalogs: true
W20260505 07:53:55.122412 1237 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260505 07:53:55.122515 1237 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260505 07:53:55.122611 1237 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260505 07:53:55.122686 1237 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260505 07:53:55.127799 1181 ts_tablet_manager.cc:1434] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.035s user 0.002s sys 0.000s
I20260505 07:53:55.130332 1181 tablet_bootstrap.cc:492] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:55.134399 1181 tablet_bootstrap.cc:654] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.136770 1172 tablet_bootstrap.cc:492] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.136957 1172 ts_tablet_manager.cc:1403] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.002s
I20260505 07:53:55.137801 1172 raft_consensus.cc:359] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.138010 1172 raft_consensus.cc:385] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.138091 1172 raft_consensus.cc:740] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.139915 1172 consensus_queue.cc:260] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.140129 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ac06bb9d273457ead843f5dd754539c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.140754 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fd990e80a90418dbb9b94cb6e2ee156 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.141320 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1496aa36a9c0478fa72f40d4595c2398"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.141863 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1496aa36a9c0478fa72f40d4595c2398 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.143931 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_9eb304c401574e9998ad4e8cb54f258b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.144416 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9eb304c401574e9998ad4e8cb54f258b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:55.141294 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 3ac06bb9d273457ead843f5dd754539c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ac06bb9d273457ead843f5dd754539c already in progress: creating tablet
I20260505 07:53:55.148475 1172 ts_tablet_manager.cc:1434] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260505 07:53:55.148837 1172 tablet_bootstrap.cc:492] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:53:55.149559 718 leader_election.cc:343] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 31b151ff45804fbaa76cfbec86c6fd7a
W20260505 07:53:55.150166 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 117720us
I20260505 07:53:55.151103 1172 tablet_bootstrap.cc:654] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.151104 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 070d1b1b30d447539911349841638c12 (table default.Table_8c90ac7761034713834e4430232a973e [id=9fcb3767fb974e7fba21836bf2a06dc6]) successfully deleted
I20260505 07:53:55.151666 1178 tablet_replica.cc:333] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.151851 1178 raft_consensus.cc:2243] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.152118 1178 raft_consensus.cc:2272] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.152086 1237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.61:33565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.0.105.61
--webserver_port=43503
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 05 May 2026 07:43:22 UTC on bdcb31816ec0
build id 11754
ASAN enabled
I20260505 07:53:55.152221 755 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4d96bdf674c2395938b9971c0ad28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e707143a4e64725aa66db05b24999a6) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.152454 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 9959be9f83c2444282d012b61eef49b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:55.152776 717 leader_election.cc:343] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.152890 1178 ts_tablet_manager.cc:1916] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.153158 717 leader_election.cc:304] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:55.153410 1273 raft_consensus.cc:2749] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:53:55.153713 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 59124.6us
I20260505 07:53:55.154071 1237 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260505 07:53:55.154430 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8087cab2da594607a0c3f7cf5839c6f3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:55.154973 717 leader_election.cc:343] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.158946 1237 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260505 07:53:55.160796 1178 ts_tablet_manager.cc:1929] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.160993 1178 log.cc:1199] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/0fd990e80a90418dbb9b94cb6e2ee156
I20260505 07:53:55.161511 1178 ts_tablet_manager.cc:1950] T 0fd990e80a90418dbb9b94cb6e2ee156 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:55.162751 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 0fd990e80a90418dbb9b94cb6e2ee156 (table default.Table_0eea08e8b83741208451d0a496b53a5c [id=40e8397fa22f44ecb51f34f6f2ac2408]) successfully deleted
I20260505 07:53:55.166455 1175 ts_tablet_manager.cc:1929] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.166594 1175 log.cc:1199] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/a25715f0fa504e92b54c48712c2c2f48
I20260505 07:53:55.167531 1175 ts_tablet_manager.cc:1950] T a25715f0fa504e92b54c48712c2c2f48 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.168447 1181 tablet_bootstrap.cc:492] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:55.168648 1181 ts_tablet_manager.cc:1403] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.038s user 0.003s sys 0.001s
I20260505 07:53:55.169504 1181 raft_consensus.cc:359] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.173291 601 master_service.cc:954] ReplaceTablet: received request to replace tablet 8573f8bd424641438e2aa6ff6869180e from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.173910 1181 raft_consensus.cc:385] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.174139 1181 raft_consensus.cc:740] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.174499 1181 consensus_queue.cc:260] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.176388 1181 ts_tablet_manager.cc:1434] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260505 07:53:55.177114 1181 tablet_bootstrap.cc:492] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:55.179297 1181 tablet_bootstrap.cc:654] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.188283 1181 tablet_bootstrap.cc:492] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:55.188540 1181 ts_tablet_manager.cc:1403] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260505 07:53:55.190119 1181 raft_consensus.cc:359] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.190388 1181 raft_consensus.cc:385] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.190524 1181 raft_consensus.cc:740] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.190855 1181 consensus_queue.cc:260] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.191560 1181 ts_tablet_manager.cc:1434] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:53:55.191895 1181 tablet_bootstrap.cc:492] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:55.197685 1181 tablet_bootstrap.cc:654] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.200627 1172 tablet_bootstrap.cc:492] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.200852 1172 ts_tablet_manager.cc:1403] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.052s user 0.000s sys 0.006s
I20260505 07:53:55.201833 1172 raft_consensus.cc:359] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.202061 1172 raft_consensus.cc:385] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.202186 1172 raft_consensus.cc:740] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.202461 1172 consensus_queue.cc:260] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.205207 1172 ts_tablet_manager.cc:1434] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260505 07:53:55.205663 1172 tablet_bootstrap.cc:492] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:53:55.208890 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2782cf753bc44bf4a102dd785ea3c6b8 consensus state has no leader
I20260505 07:53:55.210466 600 master_service.cc:954] ReplaceTablet: received request to replace tablet 2782cf753bc44bf4a102dd785ea3c6b8 from {username='slave'} at 127.0.0.1:47238
W20260505 07:53:55.210866 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8a60c015136246a3b02f70254c0bb67f has no consensus state
I20260505 07:53:55.211186 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1510db6bb88343d0aa4e95a01ecf17a6" } new_table_name: "default.Table_66ccaae918c545228101f7d0e29c37dc" modify_external_catalogs: true
I20260505 07:53:55.167739 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ac06bb9d273457ead843f5dd754539c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:55.167969 718 leader_election.cc:343] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8087cab2da594607a0c3f7cf5839c6f3
I20260505 07:53:55.211910 718 leader_election.cc:304] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:55.212296 1273 raft_consensus.cc:2749] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:53:55.212649 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 44683.2us
W20260505 07:53:55.190477 1176 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.788s user 0.204s sys 0.370s
W20260505 07:53:55.212867 1176 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.810s user 0.204s sys 0.370s
I20260505 07:53:55.214054 1176 tablet_bootstrap.cc:492] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:55.214207 1176 ts_tablet_manager.cc:1403] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.814s user 0.204s sys 0.372s
I20260505 07:53:55.214159 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_448efd98249942379556e125b25d169c" } new_table_name: "default.Table_2a2802255e0743c18185473d0dcded48" modify_external_catalogs: true
I20260505 07:53:55.214944 1176 raft_consensus.cc:359] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.215124 1176 raft_consensus.cc:385] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.215197 1176 raft_consensus.cc:740] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.215441 1176 consensus_queue.cc:260] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.215595 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_eb39510acd7a40b6a7f20bd09e300ff7" } modify_external_catalogs: true
I20260505 07:53:55.215764 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_eb39510acd7a40b6a7f20bd09e300ff7" } modify_external_catalogs: true
I20260505 07:53:55.216797 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 689792d9ea734e0bbf1df1c74dab6f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:52 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.217064 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet b2f4aff8bf0a43629e43305020df7fa3 with delete_type TABLET_DATA_DELETED (replaced by tablet 2850a301ac4f435a96742b00bf54baa9) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.217177 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6f4a022cb67a4112892d0b087d8c10b0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.217718 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f4a022cb67a4112892d0b087d8c10b0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.218343 1176 ts_tablet_manager.cc:1434] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:53:55.218863 1178 tablet_replica.cc:333] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.219075 1178 raft_consensus.cc:2243] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.219244 1178 raft_consensus.cc:2272] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.219511 1176 tablet_bootstrap.cc:492] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:53:55.220570 1292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:53:55.220996 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d75faf2076d74755af036055448af75a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.221201 1176 tablet_bootstrap.cc:654] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:55.221506 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d75faf2076d74755af036055448af75a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.222892 1175 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34194 (request call id 69) took 1027 ms. Trace:
I20260505 07:53:55.223062 1175 rpcz_store.cc:276] 0505 07:53:54.195782 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:54.197026 (+ 1244us) service_pool.cc:224] Handling call
0505 07:53:55.103608 (+906582us) ts_tablet_manager.cc:1151] Deleting tablet a25715f0fa504e92b54c48712c2c2f48
0505 07:53:55.103619 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:55.114535 (+ 10916us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.166353 (+ 51818us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.222843 (+ 56490us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":906359}
I20260505 07:53:55.223474 1175 tablet_replica.cc:333] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.223775 1175 raft_consensus.cc:2243] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.223942 1175 raft_consensus.cc:2272] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.224543 1175 ts_tablet_manager.cc:1916] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.227324 599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4d44519a55e0425d8016d1c0c28ed7f7" } modify_external_catalogs: true
I20260505 07:53:55.227519 599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4d44519a55e0425d8016d1c0c28ed7f7" } modify_external_catalogs: true
I20260505 07:53:55.229830 1178 ts_tablet_manager.cc:1916] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.230957 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet a25715f0fa504e92b54c48712c2c2f48 (table default.Table_23d3119ff64f4b9caf3ce96d7520fccf [id=0ac6c7c427534982aa4dca340c2001bc]) successfully deleted
W20260505 07:53:55.236701 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5de9e0fe8eb7477ab1cfa64b808575d8 consensus state has no leader
I20260505 07:53:55.237035 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d1ecbab8fbf047cf847ebbb4aeee4b63" } modify_external_catalogs: true
I20260505 07:53:55.237205 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d1ecbab8fbf047cf847ebbb4aeee4b63" } modify_external_catalogs: true
W20260505 07:53:55.238310 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5f57568e1c4c4573ac18d9f621eeb4c7 consensus state has no leader
I20260505 07:53:55.238559 594 master_service.cc:954] ReplaceTablet: received request to replace tablet ce9d01b0b5464a65966f26839b48aaf1 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.240612 1178 ts_tablet_manager.cc:1929] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.240813 1178 log.cc:1199] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/689792d9ea734e0bbf1df1c74dab6f83
I20260505 07:53:55.241278 1178 ts_tablet_manager.cc:1950] T 689792d9ea734e0bbf1df1c74dab6f83 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:55.242317 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 689792d9ea734e0bbf1df1c74dab6f83 (table default.Table_8b2e2e36529648cebd2bbdeda19b23b6 [id=0407ce7e0583423e92f51c513897877d]) successfully deleted
W20260505 07:53:55.242883 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5dab02a546b944f799045723a515118f consensus state has no leader
I20260505 07:53:55.246590 597 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_31dd1382f8b4494e9e13ac5b3ac18aa0 [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 3 got 0
I20260505 07:53:55.251117 592 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 2ec347e754ab4b7282e68b15a9bbf993 (table default.Table_d99fe22bedab4ff5b8635dc36e2f5060 [id=7e62bbf111c94a01b324b800bd6ced0b]). Expected version 1 got 0
I20260505 07:53:55.251354 592 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fea61fad19b34ffaa0b9df3a51979b18 (table default.Table_d0fa020605d347b3ad7096aed39c552c [id=d53c653124364ea5acc357977c9db864]). Expected version 1 got 0
I20260505 07:53:55.253096 1181 tablet_bootstrap.cc:492] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:55.253237 1181 ts_tablet_manager.cc:1403] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.001s
I20260505 07:53:55.253800 1178 tablet_replica.cc:333] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.254261 1175 ts_tablet_manager.cc:1929] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.254448 1175 log.cc:1199] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/8087cab2da594607a0c3f7cf5839c6f3
I20260505 07:53:55.254942 1175 ts_tablet_manager.cc:1950] T 8087cab2da594607a0c3f7cf5839c6f3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.255061 1178 raft_consensus.cc:2243] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.255429 1178 raft_consensus.cc:2272] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.255964 1175 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34194 (request call id 67) took 1141 ms. Trace:
I20260505 07:53:55.256101 1175 rpcz_store.cc:276] 0505 07:53:54.114434 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:54.198840 (+ 84406us) service_pool.cc:224] Handling call
0505 07:53:55.223381 (+1024541us) ts_tablet_manager.cc:1151] Deleting tablet 8087cab2da594607a0c3f7cf5839c6f3
0505 07:53:55.223393 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:55.237678 (+ 14285us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.254173 (+ 16495us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.255894 (+ 1721us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1024345}
I20260505 07:53:55.256554 1175 tablet_replica.cc:333] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.256807 1175 raft_consensus.cc:2243] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.257023 1175 raft_consensus.cc:2272] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.257647 1175 ts_tablet_manager.cc:1916] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.258155 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 202a237ca580482c98803683a77d96d2 (DEFAULT_TABLE table=default.Table_2f6f14275b8847be96f68a761156a854 [id=9c830b1f855348299c40b40610589233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.258548 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 202a237ca580482c98803683a77d96d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.259088 1178 ts_tablet_manager.cc:1916] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.259867 1181 raft_consensus.cc:359] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.264145 1181 raft_consensus.cc:385] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.264187 597 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_31dd1382f8b4494e9e13ac5b3ac18aa0 [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 3 got 0
I20260505 07:53:55.264446 1181 raft_consensus.cc:740] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.264999 1181 consensus_queue.cc:260] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.267964 1031 tablet_service.cc:1511] Processing CreateTablet for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (DEFAULT_TABLE table=default.Table_e23a9873d96a4ca8ba1bbe8c4976bc59 [id=8a910f6a553a47ea87ec170e376982d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.268313 1181 ts_tablet_manager.cc:1434] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260505 07:53:55.268457 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.268630 1181 tablet_bootstrap.cc:492] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:55.272270 1181 tablet_bootstrap.cc:654] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.272720 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet c70dedbd58ca402daf0fdde68cc5b574 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.273332 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 8087cab2da594607a0c3f7cf5839c6f3 (table default.Table_53ef916ef6d94510862d6efec4f98487 [id=2b0d644a9b4a490685837b5e427cffb8]) successfully deleted
W20260505 07:53:55.274744 1293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:53:55.274907 761 tablet_service.cc:1511] Processing CreateTablet for tablet 202a237ca580482c98803683a77d96d2 (DEFAULT_TABLE table=default.Table_2f6f14275b8847be96f68a761156a854 [id=9c830b1f855348299c40b40610589233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.275357 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 202a237ca580482c98803683a77d96d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.279675 897 tablet_service.cc:1558] Processing DeleteTablet for tablet cf95f4331e4540ae84e329cf577427b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.279696 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 222c41e6cd8b4d0093fd692618efb107 with delete_type TABLET_DATA_DELETED (replaced by tablet b404e50ae092419c9396772b3b14eeaa) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.279989 1171 tablet_replica.cc:333] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.284890 1237 server_base.cc:1061] running on GCE node
I20260505 07:53:55.285612 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4d96bdf674c2395938b9971c0ad28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e707143a4e64725aa66db05b24999a6) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.286207 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6490f568024510ab843ec5205a4d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.286348 1171 raft_consensus.cc:2243] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.286553 1171 raft_consensus.cc:2272] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.288527 1171 ts_tablet_manager.cc:1916] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.288508 898 tablet_service.cc:1511] Processing CreateTablet for tablet 202a237ca580482c98803683a77d96d2 (DEFAULT_TABLE table=default.Table_2f6f14275b8847be96f68a761156a854 [id=9c830b1f855348299c40b40610589233]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.289470 895 tablet_service.cc:1511] Processing CreateTablet for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (DEFAULT_TABLE table=default.Table_e23a9873d96a4ca8ba1bbe8c4976bc59 [id=8a910f6a553a47ea87ec170e376982d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.290081 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 62d4d96bdf674c2395938b9971c0ad28 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.291290 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 202a237ca580482c98803683a77d96d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.291383 1175 ts_tablet_manager.cc:1929] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.291513 1175 log.cc:1199] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/b7cce14283e7405690a15e26be8a45b3
I20260505 07:53:55.291466 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 3d83e9d4243e409794ffe8a342bce6b3 (DEFAULT_TABLE table=default.Table_e9ed38d78117420d98a771675ab512bb [id=5a7033e8ecc940e49794281e421153ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.291814 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d83e9d4243e409794ffe8a342bce6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.291909 1175 ts_tablet_manager.cc:1950] T b7cce14283e7405690a15e26be8a45b3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.292695 1175 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34194 (request call id 71) took 1055 ms. Trace:
I20260505 07:53:55.292800 1175 rpcz_store.cc:276] 0505 07:53:54.237439 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:54.237493 (+ 54us) service_pool.cc:224] Handling call
0505 07:53:55.256460 (+1018967us) ts_tablet_manager.cc:1151] Deleting tablet b7cce14283e7405690a15e26be8a45b3
0505 07:53:55.256473 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:55.275104 (+ 18631us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.291304 (+ 16200us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.292647 (+ 1343us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1018670}
I20260505 07:53:55.293188 1175 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34194 (request call id 76) took 1049 ms. Trace:
I20260505 07:53:55.293290 1175 rpcz_store.cc:276] 0505 07:53:54.243807 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:54.243868 (+ 61us) service_pool.cc:224] Handling call
0505 07:53:55.293051 (+1049183us) ts_tablet_manager.cc:1151] Deleting tablet c7652191b1d44ba59d856a2b0d7017b1
0505 07:53:55.293060 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:55.293162 (+ 102us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1048970}
I20260505 07:53:55.293547 1175 tablet_replica.cc:333] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.293720 1175 raft_consensus.cc:2243] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.293860 1175 raft_consensus.cc:2272] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.294548 1171 ts_tablet_manager.cc:1929] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.294663 1171 log.cc:1199] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/cf95f4331e4540ae84e329cf577427b8
I20260505 07:53:55.294998 1171 ts_tablet_manager.cc:1950] T cf95f4331e4540ae84e329cf577427b8 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.295804 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6f4a022cb67a4112892d0b087d8c10b0" } modify_external_catalogs: true
I20260505 07:53:55.295976 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6f4a022cb67a4112892d0b087d8c10b0" } modify_external_catalogs: true
I20260505 07:53:55.296758 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.299808 901 tablet_service.cc:1511] Processing CreateTablet for tablet 872362aecf6a44b59e586b1fa7637c8b (DEFAULT_TABLE table=default.Table_4d44519a55e0425d8016d1c0c28ed7f7 [id=4a43764f58e442ad892621ae1f29b214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.300211 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872362aecf6a44b59e586b1fa7637c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.300338 890 tablet_service.cc:1511] Processing CreateTablet for tablet 3d83e9d4243e409794ffe8a342bce6b3 (DEFAULT_TABLE table=default.Table_e9ed38d78117420d98a771675ab512bb [id=5a7033e8ecc940e49794281e421153ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.300673 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d83e9d4243e409794ffe8a342bce6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.302979 1175 ts_tablet_manager.cc:1916] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.303309 1171 tablet_replica.cc:333] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.303504 1171 raft_consensus.cc:2243] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.303679 1171 raft_consensus.cc:2272] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.304252 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet cf95f4331e4540ae84e329cf577427b8 (table default.Table_8a35dff4878b4429b4a6600b8259fd83 [id=0e331c73bbf842dcac6c971a6688f58b]) successfully deleted
I20260505 07:53:55.304387 1171 ts_tablet_manager.cc:1916] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.304870 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 4d6490f568024510ab843ec5205a4d1a from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.309468 1175 ts_tablet_manager.cc:1929] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.309609 1175 log.cc:1199] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/772a65f4016843d18fbca25842f76de0
I20260505 07:53:55.309986 1175 ts_tablet_manager.cc:1950] T 772a65f4016843d18fbca25842f76de0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.310384 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet b7cce14283e7405690a15e26be8a45b3 (table default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]) successfully deleted
I20260505 07:53:55.310488 755 tablet_service.cc:1511] Processing CreateTablet for tablet 3d83e9d4243e409794ffe8a342bce6b3 (DEFAULT_TABLE table=default.Table_e9ed38d78117420d98a771675ab512bb [id=5a7033e8ecc940e49794281e421153ea]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:55.310654 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet c7652191b1d44ba59d856a2b0d7017b1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c7652191b1d44ba59d856a2b0d7017b1 already in progress: creating tablet
I20260505 07:53:55.310705 1175 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34194 (request call id 75) took 1070 ms. Trace:
I20260505 07:53:55.310894 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d83e9d4243e409794ffe8a342bce6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.310801 1175 rpcz_store.cc:276] 0505 07:53:54.240087 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:54.247276 (+ 7189us) service_pool.cc:224] Handling call
0505 07:53:55.293496 (+1046220us) ts_tablet_manager.cc:1151] Deleting tablet 772a65f4016843d18fbca25842f76de0
0505 07:53:55.293504 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:55.306024 (+ 12520us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.309410 (+ 3386us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.310649 (+ 1239us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1046019}
I20260505 07:53:55.311124 750 tablet_service.cc:1511] Processing CreateTablet for tablet 872362aecf6a44b59e586b1fa7637c8b (DEFAULT_TABLE table=default.Table_4d44519a55e0425d8016d1c0c28ed7f7 [id=4a43764f58e442ad892621ae1f29b214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.311183 1171 ts_tablet_manager.cc:1929] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.311282 1171 log.cc:1199] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/222c41e6cd8b4d0093fd692618efb107
I20260505 07:53:55.311429 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872362aecf6a44b59e586b1fa7637c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.311610 1171 ts_tablet_manager.cc:1950] T 222c41e6cd8b4d0093fd692618efb107 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.312458 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 222c41e6cd8b4d0093fd692618efb107 (table default.Table_45a166ce5441483abbdc62a96ee310c2 [id=8f533b7aa2374c439f7234a373af31bb]) successfully deleted
I20260505 07:53:55.315186 1171 tablet_replica.cc:333] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.315375 1171 raft_consensus.cc:2243] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.315516 1171 raft_consensus.cc:2272] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.316499 1171 ts_tablet_manager.cc:1916] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.317662 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 872362aecf6a44b59e586b1fa7637c8b (DEFAULT_TABLE table=default.Table_4d44519a55e0425d8016d1c0c28ed7f7 [id=4a43764f58e442ad892621ae1f29b214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.318454 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 872362aecf6a44b59e586b1fa7637c8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.320459 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2a2802255e0743c18185473d0dcded48" } modify_external_catalogs: true
I20260505 07:53:55.320657 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2a2802255e0743c18185473d0dcded48" } modify_external_catalogs: true
I20260505 07:53:55.321990 1171 ts_tablet_manager.cc:1929] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.322100 1171 log.cc:1199] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/62d4d96bdf674c2395938b9971c0ad28
I20260505 07:53:55.322463 1171 ts_tablet_manager.cc:1950] T 62d4d96bdf674c2395938b9971c0ad28 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.323210 1175 tablet_replica.cc:333] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.323400 1175 raft_consensus.cc:2243] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.323395 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 62d4d96bdf674c2395938b9971c0ad28 (table default.Table_90e7403dad6348e9a3b45c777dfffa98 [id=c3a480b747de444aa45b771244908352]) successfully deleted
I20260505 07:53:55.323642 1175 raft_consensus.cc:2272] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.323652 761 tablet_service.cc:1511] Processing CreateTablet for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (DEFAULT_TABLE table=default.Table_e23a9873d96a4ca8ba1bbe8c4976bc59 [id=8a910f6a553a47ea87ec170e376982d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.324038 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.324065 1181 tablet_bootstrap.cc:492] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:55.324208 1181 ts_tablet_manager.cc:1403] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.056s user 0.003s sys 0.000s
I20260505 07:53:55.324198 1175 ts_tablet_manager.cc:1916] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:55.324199 1298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:53:55.325008 1181 raft_consensus.cc:359] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.325218 1181 raft_consensus.cc:385] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.325273 1172 tablet_bootstrap.cc:654] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.325295 1181 raft_consensus.cc:740] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.325520 1237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260505 07:53:55.325593 1181 consensus_queue.cc:260] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.327113 1181 ts_tablet_manager.cc:1434] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:55.327412 1181 tablet_bootstrap.cc:492] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:55.328406 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 8087cab2da594607a0c3f7cf5839c6f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.331948 1237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260505 07:53:55.333207 1237 hybrid_clock.cc:648] HybridClock initialized: now 1777967635333095 us; error 98 us; skew 500 ppm
I20260505 07:53:55.333712 1237 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260505 07:53:55.334908 1172 tablet_bootstrap.cc:492] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.335062 1172 ts_tablet_manager.cc:1403] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.129s user 0.003s sys 0.000s
I20260505 07:53:55.335172 1181 tablet_bootstrap.cc:654] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.335465 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1da2218f525749af888a9da48524bc11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.335603 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_55630f6d78594e3d810e3291317b348c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.335808 1172 raft_consensus.cc:359] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.335995 1172 raft_consensus.cc:385] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.336063 1172 raft_consensus.cc:740] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:53:55.336135 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55630f6d78594e3d810e3291317b348c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:55.336297 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1da2218f525749af888a9da48524bc11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.336287 1172 consensus_queue.cc:260] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.339949 750 tablet_service.cc:1511] Processing CreateTablet for tablet 1852f7f4ff594639908b03730b68fe86 (DEFAULT_TABLE table=default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.340191 890 tablet_service.cc:1511] Processing CreateTablet for tablet 1852f7f4ff594639908b03730b68fe86 (DEFAULT_TABLE table=default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.340322 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1852f7f4ff594639908b03730b68fe86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.340587 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1852f7f4ff594639908b03730b68fe86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.341423 755 tablet_service.cc:1511] Processing CreateTablet for tablet ccdde3e5a13e436d9f2328831e155adb (DEFAULT_TABLE table=default.Table_114b1df99e44400c9d52153e3c154ed1 [id=047ae44eaf5e48ba8b11e90f6d5f8119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.341858 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccdde3e5a13e436d9f2328831e155adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.343854 1035 tablet_service.cc:1511] Processing CreateTablet for tablet ccdde3e5a13e436d9f2328831e155adb (DEFAULT_TABLE table=default.Table_114b1df99e44400c9d52153e3c154ed1 [id=047ae44eaf5e48ba8b11e90f6d5f8119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.344228 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccdde3e5a13e436d9f2328831e155adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.344455 901 tablet_service.cc:1511] Processing CreateTablet for tablet ccdde3e5a13e436d9f2328831e155adb (DEFAULT_TABLE table=default.Table_114b1df99e44400c9d52153e3c154ed1 [id=047ae44eaf5e48ba8b11e90f6d5f8119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.344806 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccdde3e5a13e436d9f2328831e155adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.337800 1178 ts_tablet_manager.cc:1929] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.339707 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 1852f7f4ff594639908b03730b68fe86 (DEFAULT_TABLE table=default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.345326 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1852f7f4ff594639908b03730b68fe86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.347039 1172 ts_tablet_manager.cc:1434] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260505 07:53:55.347324 1172 tablet_bootstrap.cc:492] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.350159 1172 tablet_bootstrap.cc:654] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.351725 1171 tablet_replica.cc:333] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.351917 1171 raft_consensus.cc:2243] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.352102 1171 raft_consensus.cc:2272] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.352608 1178 log.cc:1199] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/b2f4aff8bf0a43629e43305020df7fa3
I20260505 07:53:55.353143 1178 ts_tablet_manager.cc:1950] T b2f4aff8bf0a43629e43305020df7fa3 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:55.353610 1171 ts_tablet_manager.cc:1916] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.354893 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet b2f4aff8bf0a43629e43305020df7fa3 (table default.Table_74bdaa65d97949c9ae16b85c62dbcc59 [id=d524458d873a48cb980e2f85490232d6]) successfully deleted
I20260505 07:53:55.355607 1175 ts_tablet_manager.cc:1929] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.355723 1175 log.cc:1199] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/31b151ff45804fbaa76cfbec86c6fd7a
I20260505 07:53:55.356328 1175 ts_tablet_manager.cc:1950] T 31b151ff45804fbaa76cfbec86c6fd7a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.357167 1175 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34194 (request call id 77) took 1093 ms. Trace:
I20260505 07:53:55.357300 1175 rpcz_store.cc:276] 0505 07:53:54.263929 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:54.274891 (+ 10962us) service_pool.cc:224] Handling call
0505 07:53:55.311041 (+1036150us) ts_tablet_manager.cc:1151] Deleting tablet 31b151ff45804fbaa76cfbec86c6fd7a
0505 07:53:55.311052 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:55.337572 (+ 26520us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.355502 (+ 17930us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:55.357120 (+ 1618us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1032014}
I20260505 07:53:55.357753 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 31b151ff45804fbaa76cfbec86c6fd7a (table default.Table_7f23cbfec9794a2bbc9e9dad2d07011c [id=b25c09d8fb8347af83c69e76a523867e]) successfully deleted
I20260505 07:53:55.358125 1181 tablet_bootstrap.cc:492] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:55.358323 1181 ts_tablet_manager.cc:1403] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.000s
I20260505 07:53:55.358603 1229 raft_consensus.cc:493] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:55.358763 1229 raft_consensus.cc:515] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.359411 1229 leader_election.cc:290] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:55.359284 1181 raft_consensus.cc:359] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.359619 1181 raft_consensus.cc:385] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.359764 1181 raft_consensus.cc:740] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.360046 1181 consensus_queue.cc:260] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.360082 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec347e754ab4b7282e68b15a9bbf993" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:55.360596 854 leader_election.cc:343] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.361285 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec347e754ab4b7282e68b15a9bbf993" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:55.361753 856 leader_election.cc:343] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.361881 856 leader_election.cc:304] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:55.362144 1175 tablet_replica.cc:333] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
W20260505 07:53:55.362740 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 8573f8bd424641438e2aa6ff6869180e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8573f8bd424641438e2aa6ff6869180e already in progress: creating tablet
I20260505 07:53:55.362798 1229 raft_consensus.cc:2749] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:55.363059 1229 raft_consensus.cc:493] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:55.363214 1229 raft_consensus.cc:515] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.363271 1181 ts_tablet_manager.cc:1434] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:53:55.363727 1237 webserver.cc:492] Webserver started at http://127.0.105.61:43503/ using document root <none> and password file <none>
I20260505 07:53:55.363795 1229 leader_election.cc:290] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:55.364395 1181 tablet_bootstrap.cc:492] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:55.364459 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9d01b0b5464a65966f26839b48aaf1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:55.364936 856 leader_election.cc:343] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.365036 1237 fs_manager.cc:362] Metadata directory not provided
I20260505 07:53:55.365187 1237 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260505 07:53:55.366171 1181 tablet_bootstrap.cc:654] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.367803 1178 tablet_replica.cc:333] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.367998 1178 raft_consensus.cc:2243] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.368232 1178 raft_consensus.cc:2272] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.368230 1171 ts_tablet_manager.cc:1929] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.368379 1171 log.cc:1199] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/4d6490f568024510ab843ec5205a4d1a
I20260505 07:53:55.368729 1171 ts_tablet_manager.cc:1950] T 4d6490f568024510ab843ec5205a4d1a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.369343 1178 ts_tablet_manager.cc:1916] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.369796 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 4d6490f568024510ab843ec5205a4d1a (table default.Table_ae9bf3e293014ec781c8885d8a716e83 [id=9c204b77f3df4bf9a58508967c89c9f4]) successfully deleted
I20260505 07:53:55.372329 890 tablet_service.cc:1511] Processing CreateTablet for tablet 8a60c015136246a3b02f70254c0bb67f (DEFAULT_TABLE table=default.Table_d9857ceb9b4d48dab8120e696f98e0e5 [id=65a99c8ba7b14ed0aec12382a2c8095c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.372718 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a60c015136246a3b02f70254c0bb67f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.374658 1172 tablet_bootstrap.cc:492] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.374801 1172 ts_tablet_manager.cc:1403] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.028s user 0.005s sys 0.000s
I20260505 07:53:55.374881 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 8a60c015136246a3b02f70254c0bb67f (DEFAULT_TABLE table=default.Table_d9857ceb9b4d48dab8120e696f98e0e5 [id=65a99c8ba7b14ed0aec12382a2c8095c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.375218 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a60c015136246a3b02f70254c0bb67f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.375614 1172 raft_consensus.cc:359] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.375792 1172 raft_consensus.cc:385] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.375883 1172 raft_consensus.cc:740] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.376092 1172 consensus_queue.cc:260] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.377990 1172 ts_tablet_manager.cc:1434] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:55.378283 1172 tablet_bootstrap.cc:492] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.380282 1172 tablet_bootstrap.cc:654] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.381306 1181 tablet_bootstrap.cc:492] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:55.381493 1181 ts_tablet_manager.cc:1403] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260505 07:53:55.382385 1181 raft_consensus.cc:359] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.382632 1181 raft_consensus.cc:385] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.382769 1181 raft_consensus.cc:740] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.383483 1181 consensus_queue.cc:260] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.383698 1175 raft_consensus.cc:2243] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.384251 1181 ts_tablet_manager.cc:1434] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260505 07:53:55.384642 1181 tablet_bootstrap.cc:492] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:55.386013 1175 raft_consensus.cc:2272] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.386601 1181 tablet_bootstrap.cc:654] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.387095 1175 ts_tablet_manager.cc:1916] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.387949 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 25a74b65d5d6459293ab4461f4d34cf2
I20260505 07:53:55.389287 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 872362aecf6a44b59e586b1fa7637c8b
I20260505 07:53:55.390353 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 9862b87d74974dc49d7dc9049b2a694a
I20260505 07:53:55.391116 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9d01b0b5464a65966f26839b48aaf1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:55.391779 854 leader_election.cc:343] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.391932 854 leader_election.cc:304] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:55.392135 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 872362aecf6a44b59e586b1fa7637c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.392367 1229 raft_consensus.cc:2749] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:55.392416 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 9862b87d74974dc49d7dc9049b2a694a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.392737 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 9862b87d74974dc49d7dc9049b2a694a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.393685 600 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 2782cf753bc44bf4a102dd785ea3c6b8
I20260505 07:53:55.394136 600 catalog_manager.cc:6741] ReplaceTablet: tablet 2782cf753bc44bf4a102dd785ea3c6b8 deleted and replaced by tablet 8d1ce00e487d4702a6affcad57a7fbe1
I20260505 07:53:55.396569 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 9862b87d74974dc49d7dc9049b2a694a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.396927 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 2782cf753bc44bf4a102dd785ea3c6b8 with delete_type TABLET_DATA_DELETED (replaced by tablet 8d1ce00e487d4702a6affcad57a7fbe1) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.397229 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 2782cf753bc44bf4a102dd785ea3c6b8 with delete_type TABLET_DATA_DELETED (replaced by tablet 8d1ce00e487d4702a6affcad57a7fbe1) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.397746 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 25a74b65d5d6459293ab4461f4d34cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.399223 1178 ts_tablet_manager.cc:1929] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.399957 1178 log.cc:1199] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/c70dedbd58ca402daf0fdde68cc5b574
I20260505 07:53:55.400403 1178 ts_tablet_manager.cc:1950] T c70dedbd58ca402daf0fdde68cc5b574 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:55.401446 1178 tablet_replica.cc:333] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.401546 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet c70dedbd58ca402daf0fdde68cc5b574 (table default.Table_2dba7ce60556464488c3d723065cbbfd [id=b75fe4dac469427c8cf8ccd447f70973]) successfully deleted
I20260505 07:53:55.401715 1178 raft_consensus.cc:2243] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.401921 1178 raft_consensus.cc:2272] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:53:55.402459 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 9862b87d74974dc49d7dc9049b2a694a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9862b87d74974dc49d7dc9049b2a694a already in progress: creating tablet
I20260505 07:53:55.402823 1178 ts_tablet_manager.cc:1916] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:55.403717 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 2782cf753bc44bf4a102dd785ea3c6b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2782cf753bc44bf4a102dd785ea3c6b8 already in progress: creating tablet
W20260505 07:53:55.404609 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 25a74b65d5d6459293ab4461f4d34cf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25a74b65d5d6459293ab4461f4d34cf2 already in progress: creating tablet
I20260505 07:53:55.407104 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 2782cf753bc44bf4a102dd785ea3c6b8 with delete_type TABLET_DATA_DELETED (replaced by tablet 8d1ce00e487d4702a6affcad57a7fbe1) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.407682 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 25a74b65d5d6459293ab4461f4d34cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.410723 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet ce9d01b0b5464a65966f26839b48aaf1
I20260505 07:53:55.410835 1178 ts_tablet_manager.cc:1929] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.410930 1178 log.cc:1199] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/8087cab2da594607a0c3f7cf5839c6f3
I20260505 07:53:55.411278 1178 ts_tablet_manager.cc:1950] T 8087cab2da594607a0c3f7cf5839c6f3 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:55.411314 594 catalog_manager.cc:6741] ReplaceTablet: tablet ce9d01b0b5464a65966f26839b48aaf1 deleted and replaced by tablet 17327cee69a24e0cb493ff55f948bf6c
I20260505 07:53:55.411409 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 872362aecf6a44b59e586b1fa7637c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.412786 595 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 1672c633214647acbb7074a4cf018d35
I20260505 07:53:55.413796 890 tablet_service.cc:1558] Processing DeleteTablet for tablet ce9d01b0b5464a65966f26839b48aaf1 with delete_type TABLET_DATA_DELETED (replaced by tablet 17327cee69a24e0cb493ff55f948bf6c) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.414053 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.415652 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 5dab02a546b944f799045723a515118f
I20260505 07:53:55.416214 750 tablet_service.cc:1511] Processing CreateTablet for tablet 8a60c015136246a3b02f70254c0bb67f (DEFAULT_TABLE table=default.Table_d9857ceb9b4d48dab8120e696f98e0e5 [id=65a99c8ba7b14ed0aec12382a2c8095c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.418747 1171 tablet_replica.cc:333] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.419047 1171 raft_consensus.cc:2243] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.419219 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 5dab02a546b944f799045723a515118f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.419275 1171 raft_consensus.cc:2272] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.420169 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 5dab02a546b944f799045723a515118f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.422793 1229 raft_consensus.cc:493] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:55.422956 1229 raft_consensus.cc:515] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.423053 752 tablet_service.cc:1558] Processing DeleteTablet for tablet ce9d01b0b5464a65966f26839b48aaf1 with delete_type TABLET_DATA_DELETED (replaced by tablet 17327cee69a24e0cb493ff55f948bf6c) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.423219 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a60c015136246a3b02f70254c0bb67f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.423633 1229 leader_election.cc:290] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:55.423879 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d27b912f6fe14120afb18354e772a658"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.424365 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cfe212cb6c14ce3a63b6ee4275f66f1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:55.424489 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d27b912f6fe14120afb18354e772a658 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:55.424942 854 leader_election.cc:343] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.426429 1237 fs_manager.cc:714] Time spent opening directory manager: real 0.059s user 0.005s sys 0.002s
I20260505 07:53:55.428592 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_45a166ce5441483abbdc62a96ee310c2" } new_table_name: "default.Table_ff28fa06b61e4bcb9f00bfd8bcc0e447" modify_external_catalogs: true
I20260505 07:53:55.431277 1175 ts_tablet_manager.cc:1929] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.431434 1175 log.cc:1199] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/05b8c0a2185e473cbb4165fed9e94c8a
I20260505 07:53:55.431869 1175 ts_tablet_manager.cc:1950] T 05b8c0a2185e473cbb4165fed9e94c8a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.432611 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_e0534e758791440aa17726afbfbf40aa" } new_table_name: "default.Table_cb7435665bdd45ff8f517372a18d10b6" modify_external_catalogs: true
I20260505 07:53:55.433655 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 05b8c0a2185e473cbb4165fed9e94c8a (table default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]) successfully deleted
W20260505 07:53:55.433905 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 8713b273c7b8438d8749f595ee5f1629 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8713b273c7b8438d8749f595ee5f1629 already in progress: creating tablet
W20260505 07:53:55.434072 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet faf1140fe05e4f9493563ab8464c582b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet faf1140fe05e4f9493563ab8464c582b already in progress: creating tablet
W20260505 07:53:55.434265 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 09a8dbb7fe4b406d804c0b62ba006adb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09a8dbb7fe4b406d804c0b62ba006adb already in progress: creating tablet
W20260505 07:53:55.434448 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 1765a95cc3414c73b7e3918913a35937 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1765a95cc3414c73b7e3918913a35937 already in progress: creating tablet
W20260505 07:53:55.434695 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 3272555a8c814c0ebe61ca32d715212b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3272555a8c814c0ebe61ca32d715212b already in progress: creating tablet
W20260505 07:53:55.434883 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 451e99a0dfec4823812e5991147b132e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 451e99a0dfec4823812e5991147b132e already in progress: creating tablet
W20260505 07:53:55.435071 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 62d4d96bdf674c2395938b9971c0ad28 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62d4d96bdf674c2395938b9971c0ad28 already in progress: creating tablet
W20260505 07:53:55.435235 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 9959be9f83c2444282d012b61eef49b5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9959be9f83c2444282d012b61eef49b5 already in progress: creating tablet
W20260505 07:53:55.435501 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 3ac06bb9d273457ead843f5dd754539c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ac06bb9d273457ead843f5dd754539c already in progress: creating tablet
I20260505 07:53:55.435801 1171 ts_tablet_manager.cc:1916] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.436456 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_4814c9f438cd47658c1645a4fe092e1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.437103 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4814c9f438cd47658c1645a4fe092e1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.440160 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_5cd903011cdc44a392680507ed9088c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.440713 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5cd903011cdc44a392680507ed9088c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:55.442726 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 9862b87d74974dc49d7dc9049b2a694a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9862b87d74974dc49d7dc9049b2a694a already in progress: creating tablet
I20260505 07:53:55.413589 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet ce9d01b0b5464a65966f26839b48aaf1 with delete_type TABLET_DATA_DELETED (replaced by tablet 17327cee69a24e0cb493ff55f948bf6c) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.417011 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 5dab02a546b944f799045723a515118f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.426651 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cfe212cb6c14ce3a63b6ee4275f66f1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:55.444417 1176 tablet_bootstrap.cc:492] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:55.444571 1176 ts_tablet_manager.cc:1403] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.225s user 0.013s sys 0.004s
W20260505 07:53:55.445401 856 leader_election.cc:343] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.445531 856 leader_election.cc:304] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:55.446485 1312 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260505 07:53:55.446895 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 2782cf753bc44bf4a102dd785ea3c6b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2782cf753bc44bf4a102dd785ea3c6b8 already in progress: creating tablet
I20260505 07:53:55.446027 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 8087cab2da594607a0c3f7cf5839c6f3 (table default.Table_53ef916ef6d94510862d6efec4f98487 [id=2b0d644a9b4a490685837b5e427cffb8]) successfully deleted
I20260505 07:53:55.447503 1229 raft_consensus.cc:2749] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:53:55.447665 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 872362aecf6a44b59e586b1fa7637c8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 872362aecf6a44b59e586b1fa7637c8b already in progress: creating tablet
W20260505 07:53:55.447870 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 9862b87d74974dc49d7dc9049b2a694a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9862b87d74974dc49d7dc9049b2a694a already in progress: creating tablet
W20260505 07:53:55.448130 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 2782cf753bc44bf4a102dd785ea3c6b8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2782cf753bc44bf4a102dd785ea3c6b8 already in progress: creating tablet
W20260505 07:53:55.448325 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet ce9d01b0b5464a65966f26839b48aaf1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce9d01b0b5464a65966f26839b48aaf1 already in progress: creating tablet
I20260505 07:53:55.448488 1237 fs_manager.cc:730] Time spent opening block manager: real 0.013s user 0.000s sys 0.003s
W20260505 07:53:55.448557 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 5dab02a546b944f799045723a515118f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5dab02a546b944f799045723a515118f already in progress: creating tablet
I20260505 07:53:55.448887 1176 raft_consensus.cc:359] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:55.449120 1176 raft_consensus.cc:385] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.449215 1176 raft_consensus.cc:740] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.449981 1171 ts_tablet_manager.cc:1929] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.450193 1171 log.cc:1199] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/ce9d01b0b5464a65966f26839b48aaf1
I20260505 07:53:55.448666 1237 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data,/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal
uuid: "23c57a2c927c4c54bccecfeb3a775311"
format_stamp: "Formatted at 2026-05-05 07:53:50 on dist-test-slave-ksjb"
I20260505 07:53:55.450322 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_df22234f7f8f4f7d8e566c12bc5aa0a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.450555 1171 ts_tablet_manager.cc:1950] T ce9d01b0b5464a65966f26839b48aaf1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.450443 1237 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260505 07:53:55.450806 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df22234f7f8f4f7d8e566c12bc5aa0a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.451442 1171 tablet_replica.cc:333] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.451555 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet ce9d01b0b5464a65966f26839b48aaf1 (table default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]) successfully deleted
I20260505 07:53:55.451673 1171 raft_consensus.cc:2243] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.451848 1171 raft_consensus.cc:2272] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.452571 1171 ts_tablet_manager.cc:1916] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:55.452958 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 25a74b65d5d6459293ab4461f4d34cf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25a74b65d5d6459293ab4461f4d34cf2 already in progress: creating tablet
I20260505 07:53:55.454257 1176 consensus_queue.cc:260] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:55.457568 1176 ts_tablet_manager.cc:1434] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20260505 07:53:55.447230 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_9912957f51d44c0ab0b2aaaf516ad36f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.458985 1171 ts_tablet_manager.cc:1929] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.459091 1171 log.cc:1199] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/8cfe212cb6c14ce3a63b6ee4275f66f1
W20260505 07:53:55.459285 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9912957f51d44c0ab0b2aaaf516ad36f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.460007 1171 ts_tablet_manager.cc:1950] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.460729 1176 tablet_bootstrap.cc:492] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:53:55.460703 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 872362aecf6a44b59e586b1fa7637c8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 872362aecf6a44b59e586b1fa7637c8b already in progress: creating tablet
I20260505 07:53:55.461002 1171 tablet_replica.cc:333] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.461308 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet a25715f0fa504e92b54c48712c2c2f48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.461611 1178 tablet_replica.cc:333] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.461843 1178 raft_consensus.cc:2243] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.462013 1178 raft_consensus.cc:2272] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.462158 1171 raft_consensus.cc:2243] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.462314 1171 raft_consensus.cc:2272] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.462734 1178 ts_tablet_manager.cc:1916] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.461171 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 (table default.Table_2b834837c1d54b429b8d9a937f56d726 [id=49a5c863608b402a9ca94c01e5a5c09f]) successfully deleted
I20260505 07:53:55.463265 1176 tablet_bootstrap.cc:654] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:55.463333 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 5dab02a546b944f799045723a515118f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5dab02a546b944f799045723a515118f already in progress: creating tablet
W20260505 07:53:55.463536 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet ce9d01b0b5464a65966f26839b48aaf1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ce9d01b0b5464a65966f26839b48aaf1 already in progress: creating tablet
I20260505 07:53:55.445235 597 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_8e821d63b81c43258a16db52d5295ad2 [id=b4934f65dfeb4a679111783301bb1fb9]). Expected version 2 got 0
I20260505 07:53:55.465550 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8e821d63b81c43258a16db52d5295ad2" } new_table_name: "default.Table_6b95372e816b4dac9493d85379e26080" modify_external_catalogs: true
I20260505 07:53:55.461512 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_042d1a9abf8948aebac31e5f4f810019"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.466964 1176 tablet_bootstrap.cc:492] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
W20260505 07:53:55.466974 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_042d1a9abf8948aebac31e5f4f810019 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.467070 1176 ts_tablet_manager.cc:1403] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:53:55.467569 1176 raft_consensus.cc:359] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.467854 1176 raft_consensus.cc:385] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.468526 1176 raft_consensus.cc:740] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.468788 1176 consensus_queue.cc:260] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.469635 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 2850a301ac4f435a96742b00bf54baa9 with delete_type TABLET_DATA_DELETED (replaced by tablet 956b5e05bda648349ba26d5c1ea3b279) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.469878 1175 tablet_replica.cc:333] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.470021 1175 raft_consensus.cc:2243] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.470155 1175 raft_consensus.cc:2272] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.470729 1175 ts_tablet_manager.cc:1916] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.471261 1178 ts_tablet_manager.cc:1929] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.471381 1178 log.cc:1199] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/a25715f0fa504e92b54c48712c2c2f48
I20260505 07:53:55.471521 1176 ts_tablet_manager.cc:1434] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260505 07:53:55.471740 1178 ts_tablet_manager.cc:1950] T a25715f0fa504e92b54c48712c2c2f48 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:55.473325 1171 ts_tablet_manager.cc:1916] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.474099 1176 tablet_bootstrap.cc:492] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:55.475804 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet a25715f0fa504e92b54c48712c2c2f48 (table default.Table_23d3119ff64f4b9caf3ce96d7520fccf [id=0ac6c7c427534982aa4dca340c2001bc]) successfully deleted
I20260505 07:53:55.458855 593 master_service.cc:954] ReplaceTablet: received request to replace tablet fea61fad19b34ffaa0b9df3a51979b18 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.477391 1176 tablet_bootstrap.cc:654] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.482354 1175 ts_tablet_manager.cc:1929] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.482831 1171 ts_tablet_manager.cc:1929] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.482996 1171 log.cc:1199] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/5dab02a546b944f799045723a515118f
I20260505 07:53:55.483021 1175 log.cc:1199] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/2850a301ac4f435a96742b00bf54baa9
I20260505 07:53:55.484021 1175 ts_tablet_manager.cc:1950] T 2850a301ac4f435a96742b00bf54baa9 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.484551 1171 ts_tablet_manager.cc:1950] T 5dab02a546b944f799045723a515118f P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.485600 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 5dab02a546b944f799045723a515118f (table default.Table_2a2802255e0743c18185473d0dcded48 [id=b2adaf7422c7403c8b331c0d33f0ad53]) successfully deleted
I20260505 07:53:55.485761 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 2850a301ac4f435a96742b00bf54baa9 (table default.Table_74bdaa65d97949c9ae16b85c62dbcc59 [id=d524458d873a48cb980e2f85490232d6]) successfully deleted
I20260505 07:53:55.488909 1176 tablet_bootstrap.cc:492] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:55.489068 1176 ts_tablet_manager.cc:1403] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260505 07:53:55.490078 1176 raft_consensus.cc:359] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.490265 1176 raft_consensus.cc:385] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.490356 1176 raft_consensus.cc:740] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.491246 1176 consensus_queue.cc:260] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.495036 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 25a74b65d5d6459293ab4461f4d34cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.495987 1181 tablet_bootstrap.cc:492] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
W20260505 07:53:55.496003 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 25a74b65d5d6459293ab4461f4d34cf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25a74b65d5d6459293ab4461f4d34cf2 already in progress: creating tablet
I20260505 07:53:55.496138 1181 ts_tablet_manager.cc:1403] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.112s user 0.011s sys 0.019s
I20260505 07:53:55.496798 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 09a8dbb7fe4b406d804c0b62ba006adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.497301 1175 tablet_replica.cc:333] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.497447 1181 raft_consensus.cc:359] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.497712 1181 raft_consensus.cc:385] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.497815 1181 raft_consensus.cc:740] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.497668 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 872362aecf6a44b59e586b1fa7637c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.498103 1181 consensus_queue.cc:260] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:55.498534 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 872362aecf6a44b59e586b1fa7637c8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 872362aecf6a44b59e586b1fa7637c8b already in progress: creating tablet
I20260505 07:53:55.498893 1181 ts_tablet_manager.cc:1434] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260505 07:53:55.499185 1181 tablet_bootstrap.cc:492] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:55.499388 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 05b8c0a2185e473cbb4165fed9e94c8a with delete_type TABLET_DATA_DELETED (replaced by tablet 5f57568e1c4c4573ac18d9f621eeb4c7) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.499678 1172 tablet_bootstrap.cc:492] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.499771 1175 raft_consensus.cc:2243] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.499850 1172 ts_tablet_manager.cc:1403] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.122s user 0.034s sys 0.026s
I20260505 07:53:55.499907 1178 tablet_replica.cc:333] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.499949 1175 raft_consensus.cc:2272] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.500061 1178 raft_consensus.cc:2243] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.500214 1178 raft_consensus.cc:2272] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.500526 1175 ts_tablet_manager.cc:1916] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.500664 1172 raft_consensus.cc:359] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.500857 1172 raft_consensus.cc:385] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.500926 1172 raft_consensus.cc:740] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.501155 1172 consensus_queue.cc:260] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.504905 1178 ts_tablet_manager.cc:1916] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.506217 1175 ts_tablet_manager.cc:1929] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.506335 1175 log.cc:1199] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/09a8dbb7fe4b406d804c0b62ba006adb
I20260505 07:53:55.506623 1175 ts_tablet_manager.cc:1950] T 09a8dbb7fe4b406d804c0b62ba006adb P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.507510 1176 ts_tablet_manager.cc:1434] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.018s user 0.001s sys 0.002s
W20260505 07:53:55.509446 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b404e50ae092419c9396772b3b14eeaa consensus state has no leader
I20260505 07:53:55.510476 1176 tablet_bootstrap.cc:492] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:55.510839 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_74bdaa65d97949c9ae16b85c62dbcc59" } new_table_name: "default.Table_017976a3a6af42c9a9fc722661286c0e" modify_external_catalogs: true
W20260505 07:53:55.511828 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 976359601ad74b149696dc7f555b550c has no consensus state
I20260505 07:53:55.512074 1176 tablet_bootstrap.cc:654] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.512109 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d99fe22bedab4ff5b8635dc36e2f5060" } new_table_name: "default.Table_4d8c9530e5904e54ac3bdb2a82083180" modify_external_catalogs: true
I20260505 07:53:55.513415 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_8cdf9d3d3e6349f4b8d3536f78451100"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.513876 1172 ts_tablet_manager.cc:1434] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.014s user 0.002s sys 0.000s
W20260505 07:53:55.513921 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8cdf9d3d3e6349f4b8d3536f78451100 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.514087 1181 tablet_bootstrap.cc:654] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.514185 1172 tablet_bootstrap.cc:492] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.515237 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.515702 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.515764 1172 tablet_bootstrap.cc:654] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.518214 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 09a8dbb7fe4b406d804c0b62ba006adb (table default.Table_eae91d411d9149658200d44f4837f486 [id=789cc9783535475e9929332f36141e27]) successfully deleted
I20260505 07:53:55.523168 1176 tablet_bootstrap.cc:492] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:55.523303 1176 ts_tablet_manager.cc:1403] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.001s
I20260505 07:53:55.524206 1172 tablet_bootstrap.cc:492] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.524173 1176 raft_consensus.cc:359] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.524376 1172 ts_tablet_manager.cc:1403] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.008s
I20260505 07:53:55.524390 1176 raft_consensus.cc:385] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.524456 1176 raft_consensus.cc:740] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.524717 1176 consensus_queue.cc:260] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.525105 1172 raft_consensus.cc:359] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.525281 1172 raft_consensus.cc:385] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.525364 1172 raft_consensus.cc:740] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.525624 1172 consensus_queue.cc:260] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.526618 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 8713b273c7b8438d8749f595ee5f1629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.527151 1176 ts_tablet_manager.cc:1434] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:53:55.527138 599 master_service.cc:954] ReplaceTablet: received request to replace tablet 3272555a8c814c0ebe61ca32d715212b from {username='slave'} at 127.0.0.1:47238
W20260505 07:53:55.528379 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3781d0bbcd1b4189bd836eeae4236f40 consensus state has no leader
I20260505 07:53:55.528558 1176 tablet_bootstrap.cc:492] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:55.528796 597 master_service.cc:954] ReplaceTablet: received request to replace tablet fea61fad19b34ffaa0b9df3a51979b18 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.530047 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet a25715f0fa504e92b54c48712c2c2f48 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.530818 1178 ts_tablet_manager.cc:1929] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.530977 1178 log.cc:1199] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/05b8c0a2185e473cbb4165fed9e94c8a
I20260505 07:53:55.531030 1176 tablet_bootstrap.cc:654] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.532433 1181 tablet_bootstrap.cc:492] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:55.532608 1181 ts_tablet_manager.cc:1403] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.007s
I20260505 07:53:55.533272 1181 raft_consensus.cc:359] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.533469 1181 raft_consensus.cc:385] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.533551 1181 raft_consensus.cc:740] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.533684 1178 ts_tablet_manager.cc:1950] T 05b8c0a2185e473cbb4165fed9e94c8a P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:55.533735 1172 ts_tablet_manager.cc:1434] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260505 07:53:55.533800 1181 consensus_queue.cc:260] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.534467 1181 ts_tablet_manager.cc:1434] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:53:55.534721 1181 tablet_bootstrap.cc:492] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:55.534839 1172 tablet_bootstrap.cc:492] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.536229 1181 tablet_bootstrap.cc:654] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.537122 1172 tablet_bootstrap.cc:654] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.541673 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet edb76b7961844443a9b41353c2d77d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.546813 1181 tablet_bootstrap.cc:492] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:55.546973 1181 ts_tablet_manager.cc:1403] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260505 07:53:55.547644 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet fea61fad19b34ffaa0b9df3a51979b18
I20260505 07:53:55.547819 1181 raft_consensus.cc:359] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.547920 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 05b8c0a2185e473cbb4165fed9e94c8a (table default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]) successfully deleted
I20260505 07:53:55.547984 1181 raft_consensus.cc:385] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.548069 1181 raft_consensus.cc:740] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.548156 593 catalog_manager.cc:6741] ReplaceTablet: tablet fea61fad19b34ffaa0b9df3a51979b18 deleted and replaced by tablet e278b86e7b5d444ebad8d0f6921d4c62
I20260505 07:53:55.548342 1181 consensus_queue.cc:260] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.548527 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet fea61fad19b34ffaa0b9df3a51979b18 with delete_type TABLET_DATA_DELETED (replaced by tablet e278b86e7b5d444ebad8d0f6921d4c62) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:55.549018 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2ec347e754ab4b7282e68b15a9bbf993 consensus state has no leader
I20260505 07:53:55.549335 1181 ts_tablet_manager.cc:1434] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:53:55.549599 1181 tablet_bootstrap.cc:492] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:55.552381 1181 tablet_bootstrap.cc:654] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.553357 752 tablet_service.cc:1558] Processing DeleteTablet for tablet fea61fad19b34ffaa0b9df3a51979b18 with delete_type TABLET_DATA_DELETED (replaced by tablet e278b86e7b5d444ebad8d0f6921d4c62) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.553588 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 09a8dbb7fe4b406d804c0b62ba006adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.553668 1175 tablet_replica.cc:333] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.553861 1175 raft_consensus.cc:2243] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.553884 761 tablet_service.cc:1558] Processing DeleteTablet for tablet fea61fad19b34ffaa0b9df3a51979b18 with delete_type TABLET_DATA_DELETED (replaced by tablet e278b86e7b5d444ebad8d0f6921d4c62) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.554073 1175 raft_consensus.cc:2272] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.554101 755 tablet_service.cc:1558] Processing DeleteTablet for tablet 370be7a513ee49028c8d0662b43c8d91 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dab02a546b944f799045723a515118f) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.554785 1175 ts_tablet_manager.cc:1916] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.559456 901 tablet_service.cc:1558] Processing DeleteTablet for tablet fea61fad19b34ffaa0b9df3a51979b18 with delete_type TABLET_DATA_DELETED (replaced by tablet e278b86e7b5d444ebad8d0f6921d4c62) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.559914 1171 tablet_replica.cc:333] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.560105 1171 raft_consensus.cc:2243] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.560529 1171 raft_consensus.cc:2272] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.561131 1171 ts_tablet_manager.cc:1916] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.561717 1175 ts_tablet_manager.cc:1929] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.561838 1175 log.cc:1199] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/fea61fad19b34ffaa0b9df3a51979b18
I20260505 07:53:55.562158 1175 ts_tablet_manager.cc:1950] T fea61fad19b34ffaa0b9df3a51979b18 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.562383 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 370be7a513ee49028c8d0662b43c8d91 with delete_type TABLET_DATA_DELETED (replaced by tablet 5dab02a546b944f799045723a515118f) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.563081 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet fea61fad19b34ffaa0b9df3a51979b18 (table default.Table_d0fa020605d347b3ad7096aed39c552c [id=d53c653124364ea5acc357977c9db864]) successfully deleted
I20260505 07:53:55.563377 1282 raft_consensus.cc:493] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:55.563508 1282 raft_consensus.cc:515] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.564062 1178 tablet_replica.cc:333] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.564287 1282 leader_election.cc:290] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:55.564867 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "370be7a513ee49028c8d0662b43c8d91" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:55.565073 782 raft_consensus.cc:2468] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:53:55.565074 991 leader_election.cc:343] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 370be7a513ee49028c8d0662b43c8d91
I20260505 07:53:55.565346 1178 raft_consensus.cc:2243] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.565434 988 leader_election.cc:304] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:53:55.565508 1178 raft_consensus.cc:2272] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.565963 1282 raft_consensus.cc:2804] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:55.566071 1282 raft_consensus.cc:493] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:55.566147 1282 raft_consensus.cc:3060] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:55.566224 1178 ts_tablet_manager.cc:1916] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.569087 1282 raft_consensus.cc:515] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.570062 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "370be7a513ee49028c8d0662b43c8d91" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:53:55.570238 782 raft_consensus.cc:3060] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:53:55.571272 991 leader_election.cc:343] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 370be7a513ee49028c8d0662b43c8d91
I20260505 07:53:55.573346 782 raft_consensus.cc:2468] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:53:55.573848 988 leader_election.cc:304] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:53:55.575199 1171 ts_tablet_manager.cc:1929] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.575314 1171 log.cc:1199] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/fea61fad19b34ffaa0b9df3a51979b18
I20260505 07:53:55.575657 1282 leader_election.cc:290] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:55.575753 1171 ts_tablet_manager.cc:1950] T fea61fad19b34ffaa0b9df3a51979b18 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.576009 1282 raft_consensus.cc:2804] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:55.576328 1282 raft_consensus.cc:697] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:53:55.576670 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet fea61fad19b34ffaa0b9df3a51979b18 (table default.Table_d0fa020605d347b3ad7096aed39c552c [id=d53c653124364ea5acc357977c9db864]) successfully deleted
I20260505 07:53:55.577143 1282 consensus_queue.cc:237] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:55.580989 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 956b5e05bda648349ba26d5c1ea3b279 consensus state has no leader
I20260505 07:53:55.582440 1229 raft_consensus.cc:493] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:55.582610 1229 raft_consensus.cc:515] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.583204 1229 leader_election.cc:290] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:53:55.584576 585 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 09a8dbb7fe4b406d804c0b62ba006adb because the tablet was not found. No further retry: Not found: Tablet not found: 09a8dbb7fe4b406d804c0b62ba006adb
W20260505 07:53:55.585412 585 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet fea61fad19b34ffaa0b9df3a51979b18 because the tablet was not found. No further retry: Not found: Tablet not found: fea61fad19b34ffaa0b9df3a51979b18
I20260505 07:53:55.586089 1175 tablet_replica.cc:333] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.586272 1175 raft_consensus.cc:2243] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.586421 1175 raft_consensus.cc:2272] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.588294 1175 ts_tablet_manager.cc:1916] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.591141 1178 ts_tablet_manager.cc:1929] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.591512 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3781d0bbcd1b4189bd836eeae4236f40" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:55.591899 1178 log.cc:1199] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/8713b273c7b8438d8749f595ee5f1629
W20260505 07:53:55.592133 854 leader_election.cc:343] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.592460 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 2782cf753bc44bf4a102dd785ea3c6b8 with delete_type TABLET_DATA_DELETED (replaced by tablet 8d1ce00e487d4702a6affcad57a7fbe1) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.592623 1178 ts_tablet_manager.cc:1950] T 8713b273c7b8438d8749f595ee5f1629 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:55.593952 1178 tablet_replica.cc:333] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.594172 1178 raft_consensus.cc:2243] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.594367 1178 raft_consensus.cc:2272] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.595023 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 1765a95cc3414c73b7e3918913a35937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.595603 1171 tablet_replica.cc:333] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.595834 1171 raft_consensus.cc:2243] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.596040 1171 raft_consensus.cc:2272] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.596055 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3781d0bbcd1b4189bd836eeae4236f40" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:55.596921 1172 tablet_bootstrap.cc:492] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
W20260505 07:53:55.597015 856 leader_election.cc:343] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.597239 856 leader_election.cc:304] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:55.597664 1229 raft_consensus.cc:2749] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:55.597731 1171 ts_tablet_manager.cc:1916] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.598129 1172 ts_tablet_manager.cc:1403] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.063s user 0.012s sys 0.013s
I20260505 07:53:55.599179 1172 raft_consensus.cc:359] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.599393 1172 raft_consensus.cc:385] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.599472 1172 raft_consensus.cc:740] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.599658 1178 ts_tablet_manager.cc:1916] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.599757 1172 consensus_queue.cc:260] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:55.600991 1172 ts_tablet_manager.cc:1434] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:55.601819 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4d8c9530e5904e54ac3bdb2a82083180" } modify_external_catalogs: true
I20260505 07:53:55.601999 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4d8c9530e5904e54ac3bdb2a82083180" } modify_external_catalogs: true
I20260505 07:53:55.602352 1172 tablet_bootstrap.cc:492] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.602792 600 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6ff000247d324fe8b55b1398e48cb49a" } modify_external_catalogs: true
I20260505 07:53:55.602972 600 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6ff000247d324fe8b55b1398e48cb49a" } modify_external_catalogs: true
I20260505 07:53:55.603838 601 master_service.cc:954] ReplaceTablet: received request to replace tablet c7652191b1d44ba59d856a2b0d7017b1 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.604027 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_017976a3a6af42c9a9fc722661286c0e" } modify_external_catalogs: true
I20260505 07:53:55.604197 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_017976a3a6af42c9a9fc722661286c0e" } modify_external_catalogs: true
I20260505 07:53:55.604768 594 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5f57568e1c4c4573ac18d9f621eeb4c7 (table default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]). Expected version 2 got 0
I20260505 07:53:55.605305 901 tablet_service.cc:1511] Processing CreateTablet for tablet f92c53c68a4b4e319e98b84128e8e6a4 (DEFAULT_TABLE table=default.Table_1496aa36a9c0478fa72f40d4595c2398 [id=73a1cc56dd244ba389866c21655a3a42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.606261 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92c53c68a4b4e319e98b84128e8e6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.606530 755 tablet_service.cc:1511] Processing CreateTablet for tablet f92c53c68a4b4e319e98b84128e8e6a4 (DEFAULT_TABLE table=default.Table_1496aa36a9c0478fa72f40d4595c2398 [id=73a1cc56dd244ba389866c21655a3a42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.606920 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92c53c68a4b4e319e98b84128e8e6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.606927 1172 tablet_bootstrap.cc:654] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.613678 898 tablet_service.cc:1511] Processing CreateTablet for tablet b5d113a872f9400c8f28d0239ca42377 (DEFAULT_TABLE table=default.Table_02afe59354994bffb24b190e9a7abf85 [id=7579207045ec4220b966af9016a8f050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.614063 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5d113a872f9400c8f28d0239ca42377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.614799 761 tablet_service.cc:1511] Processing CreateTablet for tablet b5d113a872f9400c8f28d0239ca42377 (DEFAULT_TABLE table=default.Table_02afe59354994bffb24b190e9a7abf85 [id=7579207045ec4220b966af9016a8f050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.615146 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5d113a872f9400c8f28d0239ca42377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.620781 594 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5f57568e1c4c4573ac18d9f621eeb4c7 (table default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]). Expected version 2 got 0
I20260505 07:53:55.621917 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f4c4ec691a3445b08b83d566bd437fb8" } modify_external_catalogs: true
I20260505 07:53:55.622078 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f4c4ec691a3445b08b83d566bd437fb8" } modify_external_catalogs: true
I20260505 07:53:55.622737 601 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4814c9f438cd47658c1645a4fe092e1a" } modify_external_catalogs: true
I20260505 07:53:55.622934 601 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4814c9f438cd47658c1645a4fe092e1a" } modify_external_catalogs: true
I20260505 07:53:55.625502 1171 ts_tablet_manager.cc:1929] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.625654 1171 log.cc:1199] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/2782cf753bc44bf4a102dd785ea3c6b8
I20260505 07:53:55.626001 1171 ts_tablet_manager.cc:1950] T 2782cf753bc44bf4a102dd785ea3c6b8 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.626952 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 2782cf753bc44bf4a102dd785ea3c6b8 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]) successfully deleted
I20260505 07:53:55.629153 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7c3d336ba1db41b9b26a2fe5475bbab0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.629635 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c3d336ba1db41b9b26a2fe5475bbab0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.630177 898 tablet_service.cc:1511] Processing CreateTablet for tablet 7585f00119e14006990ed551667b4486 (DEFAULT_TABLE table=default.Table_c61fca437ddb413e934595a23b52f40f [id=a14990bc69324c54a3c7e2b7d6116d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.630508 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7585f00119e14006990ed551667b4486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.631116 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_4060b1611196402a829ecc88ff74d21f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.631212 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 2ec347e754ab4b7282e68b15a9bbf993
W20260505 07:53:55.631678 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4060b1611196402a829ecc88ff74d21f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.632921 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ec347e754ab4b7282e68b15a9bbf993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.633140 755 tablet_service.cc:1511] Processing CreateTablet for tablet 7585f00119e14006990ed551667b4486 (DEFAULT_TABLE table=default.Table_c61fca437ddb413e934595a23b52f40f [id=a14990bc69324c54a3c7e2b7d6116d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.633534 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7585f00119e14006990ed551667b4486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.639421 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ec347e754ab4b7282e68b15a9bbf993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.640686 890 tablet_service.cc:1511] Processing CreateTablet for tablet 976359601ad74b149696dc7f555b550c (DEFAULT_TABLE table=default.Table_cb7435665bdd45ff8f517372a18d10b6 [id=0d5a336f16bb433fb2ee0f23189afc91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.640673 750 tablet_service.cc:1511] Processing CreateTablet for tablet 976359601ad74b149696dc7f555b550c (DEFAULT_TABLE table=default.Table_cb7435665bdd45ff8f517372a18d10b6 [id=0d5a336f16bb433fb2ee0f23189afc91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.641026 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 976359601ad74b149696dc7f555b550c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.641033 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 976359601ad74b149696dc7f555b550c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.643607 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 956b5e05bda648349ba26d5c1ea3b279
I20260505 07:53:55.644580 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 956b5e05bda648349ba26d5c1ea3b279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.647716 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 956b5e05bda648349ba26d5c1ea3b279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.647748 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 56cbbb89f6b84407b2c344ec62402b03 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.649335 600 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 03f31efbc98c484c950c70d5e84d2136
I20260505 07:53:55.652480 601 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet e01d4183e96c42758972af71b2a54b60
I20260505 07:53:55.652652 1273 raft_consensus.cc:493] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:55.652813 1273 raft_consensus.cc:515] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.650941 890 tablet_service.cc:1511] Processing CreateTablet for tablet 3d6494590052459485059f5dce46f34f (DEFAULT_TABLE table=default.Table_3a027f114e1e4d5fabe7ddc8f584e3ec [id=85e91b69517e415194bc6c0dd62ccf9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.653236 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 8b0cf443e3044e56a26b0a8a61c9cf7d
I20260505 07:53:55.653406 1273 leader_election.cc:290] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:55.653515 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d6494590052459485059f5dce46f34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.654036 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.654007 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca4cf248adb4f768c5586d72614ad42" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:55.654551 717 leader_election.cc:343] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.655982 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca4cf248adb4f768c5586d72614ad42" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:53:55.656049 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.656205 921 raft_consensus.cc:2468] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:53:55.656697 750 tablet_service.cc:1511] Processing CreateTablet for tablet 3d6494590052459485059f5dce46f34f (DEFAULT_TABLE table=default.Table_3a027f114e1e4d5fabe7ddc8f584e3ec [id=85e91b69517e415194bc6c0dd62ccf9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.657085 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d6494590052459485059f5dce46f34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.659564 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 56cbbb89f6b84407b2c344ec62402b03
I20260505 07:53:55.659765 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 03f31efbc98c484c950c70d5e84d2136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.659777 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 03f31efbc98c484c950c70d5e84d2136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.660153 596 catalog_manager.cc:6741] ReplaceTablet: tablet 56cbbb89f6b84407b2c344ec62402b03 deleted and replaced by tablet 9044763bed1a41739ecce4def4cca1fe
I20260505 07:53:55.660386 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 56cbbb89f6b84407b2c344ec62402b03 with delete_type TABLET_DATA_DELETED (replaced by tablet 9044763bed1a41739ecce4def4cca1fe) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.661067 901 tablet_service.cc:1511] Processing CreateTablet for tablet 7219f9896fd94187ae714635077110af (DEFAULT_TABLE table=default.Table_3c9a0977f3244728b0e4de172b1559d1 [id=d82da77797e14074b53f00e8e741744c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.661166 718 leader_election.cc:304] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:55.661319 752 tablet_service.cc:1511] Processing CreateTablet for tablet 7219f9896fd94187ae714635077110af (DEFAULT_TABLE table=default.Table_3c9a0977f3244728b0e4de172b1559d1 [id=d82da77797e14074b53f00e8e741744c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.661435 1273 raft_consensus.cc:2804] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:55.661514 1273 raft_consensus.cc:493] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:55.661628 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7219f9896fd94187ae714635077110af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.661653 1273 raft_consensus.cc:3060] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:55.661731 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7219f9896fd94187ae714635077110af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.664485 1273 raft_consensus.cc:515] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.665582 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca4cf248adb4f768c5586d72614ad42" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:53:55.666115 717 leader_election.cc:343] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.666335 1273 leader_election.cc:290] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:55.666944 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca4cf248adb4f768c5586d72614ad42" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:53:55.667146 921 raft_consensus.cc:3060] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:55.669704 921 raft_consensus.cc:2468] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:53:55.670198 718 leader_election.cc:304] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:53:55.670584 1273 raft_consensus.cc:2804] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:55.670920 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 56cbbb89f6b84407b2c344ec62402b03 with delete_type TABLET_DATA_DELETED (replaced by tablet 9044763bed1a41739ecce4def4cca1fe) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.672816 1178 ts_tablet_manager.cc:1929] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.672976 1178 log.cc:1199] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/edb76b7961844443a9b41353c2d77d5a
I20260505 07:53:55.673442 1178 ts_tablet_manager.cc:1950] T edb76b7961844443a9b41353c2d77d5a P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:55.674715 1178 tablet_replica.cc:333] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.678550 1178 raft_consensus.cc:2243] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:53:55.679057 1178 pending_rounds.cc:70] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:53:55.679275 1178 pending_rounds.cc:77] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555435332358144 op_type: NO_OP noop_request { }
I20260505 07:53:55.679462 1178 raft_consensus.cc:2889] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:53:55.679632 1178 raft_consensus.cc:2272] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.680305 1273 raft_consensus.cc:697] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:53:55.679782 901 tablet_service.cc:1511] Processing CreateTablet for tablet bd581a6799a14a8dafb80d411e8936f3 (DEFAULT_TABLE table=default.Table_9eb304c401574e9998ad4e8cb54f258b [id=76b0eb99a1f343829e5ba35de6302cf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.680620 1273 consensus_queue.cc:237] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.681092 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd581a6799a14a8dafb80d411e8936f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.683159 896 tablet_service.cc:1511] Processing CreateTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 (DEFAULT_TABLE table=default.Table_786d64e187254c7880f44b3d95290cc5 [id=2aaa5e01c641425cb50db866ad9281d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.683213 895 tablet_service.cc:1511] Processing CreateTablet for tablet fcd36b5a84b4442fb76e1610d8af4c07 (DEFAULT_TABLE table=default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.683132 897 tablet_service.cc:1511] Processing CreateTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 (DEFAULT_TABLE table=default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.683518 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909306e1e5f34b7c8fbc0d86cafce531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.683720 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd36b5a84b4442fb76e1610d8af4c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.683998 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1838f89dc14a47f99f95f4cc5fdefd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.685322 752 tablet_service.cc:1511] Processing CreateTablet for tablet fcd36b5a84b4442fb76e1610d8af4c07 (DEFAULT_TABLE table=default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.689314 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd36b5a84b4442fb76e1610d8af4c07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.692279 1172 tablet_bootstrap.cc:492] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.692860 1172 ts_tablet_manager.cc:1403] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.090s user 0.027s sys 0.032s
I20260505 07:53:55.693826 1172 raft_consensus.cc:359] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:55.694321 1172 raft_consensus.cc:385] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.694454 1172 raft_consensus.cc:740] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.694705 1172 consensus_queue.cc:260] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:55.697153 759 tablet_service.cc:1511] Processing CreateTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 (DEFAULT_TABLE table=default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.697634 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1838f89dc14a47f99f95f4cc5fdefd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.697613 761 tablet_service.cc:1511] Processing CreateTablet for tablet bd581a6799a14a8dafb80d411e8936f3 (DEFAULT_TABLE table=default.Table_9eb304c401574e9998ad4e8cb54f258b [id=76b0eb99a1f343829e5ba35de6302cf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.698055 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd581a6799a14a8dafb80d411e8936f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.698864 1172 ts_tablet_manager.cc:1434] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:53:55.703215 1172 tablet_bootstrap.cc:492] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.704849 759 tablet_service.cc:1511] Processing CreateTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 (DEFAULT_TABLE table=default.Table_786d64e187254c7880f44b3d95290cc5 [id=2aaa5e01c641425cb50db866ad9281d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.705255 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909306e1e5f34b7c8fbc0d86cafce531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.705271 1172 tablet_bootstrap.cc:654] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.708792 596 catalog_manager.cc:5671] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } }
I20260505 07:53:55.709339 596 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_31dd1382f8b4494e9e13ac5b3ac18aa0 [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 3 got 0
I20260505 07:53:55.711376 1171 tablet_replica.cc:333] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.711697 1171 raft_consensus.cc:2243] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.711918 1171 raft_consensus.cc:2272] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.712826 1171 ts_tablet_manager.cc:1916] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.713449 1172 tablet_bootstrap.cc:492] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.713624 1172 ts_tablet_manager.cc:1403] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260505 07:53:55.714411 1172 raft_consensus.cc:359] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.714602 1172 raft_consensus.cc:385] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.714684 1172 raft_consensus.cc:740] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.714947 1172 consensus_queue.cc:260] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.718413 1172 ts_tablet_manager.cc:1434] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:53:55.719725 1178 ts_tablet_manager.cc:1916] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.722465 1172 tablet_bootstrap.cc:492] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:53:55.722645 587 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.61:33565, user_credentials={real_user=slave}} blocked reactor thread for 123548us
I20260505 07:53:55.723083 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 8713b273c7b8438d8749f595ee5f1629 (table default.Table_aa53055d0748464697a41feba3c4f036 [id=af8d003d14d24f34bd57e804562d1f33]) successfully deleted
W20260505 07:53:55.723435 587 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet a25715f0fa504e92b54c48712c2c2f48 because the tablet was not found. No further retry: Not found: Tablet not found: a25715f0fa504e92b54c48712c2c2f48
I20260505 07:53:55.723716 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet edb76b7961844443a9b41353c2d77d5a (table default.Table_dda9e1fe74264c2eab25cecc65079c93 [id=50e792c4b9044685ab0605ef0a9474a5]) successfully deleted
W20260505 07:53:55.724005 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet fea61fad19b34ffaa0b9df3a51979b18 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fea61fad19b34ffaa0b9df3a51979b18 already in progress: creating tablet
I20260505 07:53:55.724149 1172 tablet_bootstrap.cc:654] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.724970 1031 tablet_service.cc:1511] Processing CreateTablet for tablet f92c53c68a4b4e319e98b84128e8e6a4 (DEFAULT_TABLE table=default.Table_1496aa36a9c0478fa72f40d4595c2398 [id=73a1cc56dd244ba389866c21655a3a42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.725376 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f92c53c68a4b4e319e98b84128e8e6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.725333 1024 tablet_service.cc:1511] Processing CreateTablet for tablet b5d113a872f9400c8f28d0239ca42377 (DEFAULT_TABLE table=default.Table_02afe59354994bffb24b190e9a7abf85 [id=7579207045ec4220b966af9016a8f050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.725533 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ec347e754ab4b7282e68b15a9bbf993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.725827 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5d113a872f9400c8f28d0239ca42377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.726177 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 976359601ad74b149696dc7f555b550c (DEFAULT_TABLE table=default.Table_cb7435665bdd45ff8f517372a18d10b6 [id=0d5a336f16bb433fb2ee0f23189afc91]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.726567 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 976359601ad74b149696dc7f555b550c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.727977 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 7585f00119e14006990ed551667b4486 (DEFAULT_TABLE table=default.Table_c61fca437ddb413e934595a23b52f40f [id=a14990bc69324c54a3c7e2b7d6116d11]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.728528 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7585f00119e14006990ed551667b4486. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.738883 1176 tablet_bootstrap.cc:492] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:55.739034 1176 ts_tablet_manager.cc:1403] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.211s user 0.017s sys 0.024s
I20260505 07:53:55.740198 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 956b5e05bda648349ba26d5c1ea3b279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.740499 1172 tablet_bootstrap.cc:492] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.740684 1172 ts_tablet_manager.cc:1403] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.005s
I20260505 07:53:55.740890 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 3d6494590052459485059f5dce46f34f (DEFAULT_TABLE table=default.Table_3a027f114e1e4d5fabe7ddc8f584e3ec [id=85e91b69517e415194bc6c0dd62ccf9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.741286 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d6494590052459485059f5dce46f34f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.741501 1172 raft_consensus.cc:359] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.741675 1172 raft_consensus.cc:385] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.741789 1172 raft_consensus.cc:740] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.742022 1172 consensus_queue.cc:260] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.743983 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 03f31efbc98c484c950c70d5e84d2136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.744066 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_bd283510c1f14a9fb29a9870d33c9971"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.744450 596 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_024c44160bb5470399f3fe342ac501bb" } new_table_name: "default.Table_422080d5c75f422fb88c26b76ffc61c4" modify_external_catalogs: true
I20260505 07:53:55.744581 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_5d02989a34e8412b819674082ae217f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.744848 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_90e7403dad6348e9a3b45c777dfffa98" } new_table_name: "default.Table_f0316c88590245aea8815b3b08e89f45" modify_external_catalogs: true
W20260505 07:53:55.745178 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d02989a34e8412b819674082ae217f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.746553 1172 ts_tablet_manager.cc:1434] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260505 07:53:55.746892 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_499d356daab040e29ca2f5eb2941d790"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.747328 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_499d356daab040e29ca2f5eb2941d790 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.747356 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 8b0cf443e3044e56a26b0a8a61c9cf7d from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.748823 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_786d64e187254c7880f44b3d95290cc5" } new_table_name: "default.Table_697f13aef35b49989ffebd1549f8e930" modify_external_catalogs: true
I20260505 07:53:55.749040 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_8fd577cca7804ca198558aa2510e53e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.749456 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8fd577cca7804ca198558aa2510e53e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.750155 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.750510 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_344c494090d44fa8b4102b525e7ca099"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.750645 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 56cbbb89f6b84407b2c344ec62402b03 with delete_type TABLET_DATA_DELETED (replaced by tablet 9044763bed1a41739ecce4def4cca1fe) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:55.750958 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_344c494090d44fa8b4102b525e7ca099 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.751240 1175 ts_tablet_manager.cc:1929] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.751288 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 7219f9896fd94187ae714635077110af (DEFAULT_TABLE table=default.Table_3c9a0977f3244728b0e4de172b1559d1 [id=d82da77797e14074b53f00e8e741744c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.751368 1175 log.cc:1199] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/370be7a513ee49028c8d0662b43c8d91
I20260505 07:53:55.751705 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7219f9896fd94187ae714635077110af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.752393 1181 tablet_bootstrap.cc:492] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:55.752545 1181 ts_tablet_manager.cc:1403] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.203s user 0.003s sys 0.000s
I20260505 07:53:55.753096 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_344daeac12fd48cfab632d6a4f90ba30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.753506 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_344daeac12fd48cfab632d6a4f90ba30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.753417 1181 raft_consensus.cc:359] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.753623 1181 raft_consensus.cc:385] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.753717 1181 raft_consensus.cc:740] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.754004 1181 consensus_queue.cc:260] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.754232 1172 tablet_bootstrap.cc:492] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.755142 1181 ts_tablet_manager.cc:1434] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260505 07:53:55.755404 1181 tablet_bootstrap.cc:492] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:53:55.756086 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7e707143a4e64725aa66db05b24999a6 consensus state has no leader
I20260505 07:53:55.757417 1181 tablet_bootstrap.cc:654] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.758348 1172 tablet_bootstrap.cc:654] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.758872 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_114b1df99e44400c9d52153e3c154ed1" } new_table_name: "default.Table_efeac090fb094bdf82f547af366d562c" modify_external_catalogs: true
W20260505 07:53:55.758958 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bd283510c1f14a9fb29a9870d33c9971 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.761372 428 master_runner.cc:186] Error getting master registration for 127.0.105.61:33565: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection refused (error 111),
I20260505 07:53:55.761503 428 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260505 07:53:55.762625 596 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e1a9420a81844d2092381123d9297661 consensus state has no leader
I20260505 07:53:55.763633 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 8573f8bd424641438e2aa6ff6869180e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.762337 1176 raft_consensus.cc:359] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.767680 1176 raft_consensus.cc:385] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.767787 1176 raft_consensus.cc:740] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.768079 1176 consensus_queue.cc:260] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.768728 1176 ts_tablet_manager.cc:1434] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260505 07:53:55.769050 1176 tablet_bootstrap.cc:492] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:55.771025 1176 tablet_bootstrap.cc:654] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.771735 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 3d6494590052459485059f5dce46f34f from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.772397 752 tablet_service.cc:1511] Processing CreateTablet for tablet 7e707143a4e64725aa66db05b24999a6 (DEFAULT_TABLE table=default.Table_f0316c88590245aea8815b3b08e89f45 [id=c3a480b747de444aa45b771244908352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.772867 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e707143a4e64725aa66db05b24999a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:55.774495 587 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9], current schema version=3) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3781d0bbcd1b4189bd836eeae4236f40 consensus state has no leader
I20260505 07:53:55.774652 898 tablet_service.cc:1511] Processing CreateTablet for tablet 7e707143a4e64725aa66db05b24999a6 (DEFAULT_TABLE table=default.Table_f0316c88590245aea8815b3b08e89f45 [id=c3a480b747de444aa45b771244908352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.775309 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e707143a4e64725aa66db05b24999a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:55.778797 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 909306e1e5f34b7c8fbc0d86cafce531 consensus state has no leader
I20260505 07:53:55.779976 1176 tablet_bootstrap.cc:492] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:55.780123 1176 ts_tablet_manager.cc:1403] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260505 07:53:55.780972 1176 raft_consensus.cc:359] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.781150 1176 raft_consensus.cc:385] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.781337 1176 raft_consensus.cc:740] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.781601 1176 consensus_queue.cc:260] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.782231 1176 ts_tablet_manager.cc:1434] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:53:55.782537 1176 tablet_bootstrap.cc:492] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:55.784639 1176 tablet_bootstrap.cc:654] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.784895 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_cf3b3e8080ba449a9f92b5175e2193b1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.785396 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf3b3e8080ba449a9f92b5175e2193b1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.785899 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1e4ca2caf1834843a4b8fc347fc080bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.786515 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e4ca2caf1834843a4b8fc347fc080bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.787276 1229 raft_consensus.cc:493] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:55.787441 1229 raft_consensus.cc:515] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.788245 1229 leader_election.cc:290] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:55.789062 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de9e0fe8eb7477ab1cfa64b808575d8" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:55.789588 567 master_runner.cc:186] Error getting master registration for 127.0.105.61:33565: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection refused (error 111),
W20260505 07:53:55.789633 854 leader_election.cc:343] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.789740 567 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260505 07:53:55.790720 755 tablet_service.cc:1558] Processing DeleteTablet for tablet edb76b7961844443a9b41353c2d77d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.791464 1172 tablet_bootstrap.cc:492] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.791689 1172 ts_tablet_manager.cc:1403] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.038s user 0.006s sys 0.002s
I20260505 07:53:55.791726 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 8713b273c7b8438d8749f595ee5f1629 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.791936 593 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_df22234f7f8f4f7d8e566c12bc5aa0a2" } new_table_name: "default.Table_4b3acc65322b4dfe93c4f1d049b573e5" modify_external_catalogs: true
I20260505 07:53:55.792407 1172 raft_consensus.cc:359] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.792577 1172 raft_consensus.cc:385] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.792637 1172 raft_consensus.cc:740] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.792840 1172 consensus_queue.cc:260] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.793346 1172 ts_tablet_manager.cc:1434] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:53:55.794117 1172 tablet_bootstrap.cc:492] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.796271 1172 tablet_bootstrap.cc:654] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:55.798880 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ccdde3e5a13e436d9f2328831e155adb consensus state has no leader
W20260505 07:53:55.802285 593 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9a06f77255544d7dabb71a4f70ada809 has no consensus state
I20260505 07:53:55.805042 1172 tablet_bootstrap.cc:492] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.805188 1172 ts_tablet_manager.cc:1403] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.004s
I20260505 07:53:55.805701 1171 ts_tablet_manager.cc:1929] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.805851 1171 log.cc:1199] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/1765a95cc3414c73b7e3918913a35937
I20260505 07:53:55.806034 1172 raft_consensus.cc:359] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.806219 1172 raft_consensus.cc:385] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.806217 1171 ts_tablet_manager.cc:1950] T 1765a95cc3414c73b7e3918913a35937 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.806380 1172 raft_consensus.cc:740] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.807520 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 956b5e05bda648349ba26d5c1ea3b279 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.807837 1175 ts_tablet_manager.cc:1950] T 370be7a513ee49028c8d0662b43c8d91 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.808573 1237 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260505 07:53:55.806921 1172 consensus_queue.cc:260] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.809141 1172 ts_tablet_manager.cc:1434] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:53:55.809867 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 1765a95cc3414c73b7e3918913a35937 (table default.Table_a006de2dda30423f83e51da1e0477f9d [id=13ca7143dd0249d597c79bbfc17ae7f9]) successfully deleted
I20260505 07:53:55.809913 1172 tablet_bootstrap.cc:492] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:55.809636 1171 tablet_replica.cc:333] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.811096 1171 raft_consensus.cc:2243] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.811299 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 370be7a513ee49028c8d0662b43c8d91 (table default.Table_2a2802255e0743c18185473d0dcded48 [id=b2adaf7422c7403c8b331c0d33f0ad53]) successfully deleted
I20260505 07:53:55.811323 1171 raft_consensus.cc:2272] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.812178 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 451e99a0dfec4823812e5991147b132e with delete_type TABLET_DATA_DELETED (replaced by tablet fcd36b5a84b4442fb76e1610d8af4c07) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.814879 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de9e0fe8eb7477ab1cfa64b808575d8" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:55.817087 856 leader_election.cc:343] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:55.817242 856 leader_election.cc:304] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:55.817665 1171 ts_tablet_manager.cc:1916] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.820969 1229 raft_consensus.cc:2749] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:55.822316 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 (DEFAULT_TABLE table=default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.822839 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 9862b87d74974dc49d7dc9049b2a694a from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.821702 1172 tablet_bootstrap.cc:654] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.823547 890 tablet_service.cc:1558] Processing DeleteTablet for tablet c7652191b1d44ba59d856a2b0d7017b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.828580 1035 tablet_service.cc:1511] Processing CreateTablet for tablet bd581a6799a14a8dafb80d411e8936f3 (DEFAULT_TABLE table=default.Table_9eb304c401574e9998ad4e8cb54f258b [id=76b0eb99a1f343829e5ba35de6302cf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.830525 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 (DEFAULT_TABLE table=default.Table_786d64e187254c7880f44b3d95290cc5 [id=2aaa5e01c641425cb50db866ad9281d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.831197 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1838f89dc14a47f99f95f4cc5fdefd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.831162 1178 ts_tablet_manager.cc:1929] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:53:55.832542 1237 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260505 07:53:55.832861 1237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260505 07:53:55.833828 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d9857ceb9b4d48dab8120e696f98e0e5" } modify_external_catalogs: true
I20260505 07:53:55.834017 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d9857ceb9b4d48dab8120e696f98e0e5" } modify_external_catalogs: true
I20260505 07:53:55.834765 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 31b151ff45804fbaa76cfbec86c6fd7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.836373 1178 log.cc:1199] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/370be7a513ee49028c8d0662b43c8d91
I20260505 07:53:55.836407 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 7e707143a4e64725aa66db05b24999a6 (DEFAULT_TABLE table=default.Table_f0316c88590245aea8815b3b08e89f45 [id=c3a480b747de444aa45b771244908352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.837271 1024 tablet_service.cc:1511] Processing CreateTablet for tablet fcd36b5a84b4442fb76e1610d8af4c07 (DEFAULT_TABLE table=default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.837677 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bd581a6799a14a8dafb80d411e8936f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.837888 1178 ts_tablet_manager.cc:1950] T 370be7a513ee49028c8d0662b43c8d91 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:55.838203 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcd36b5a84b4442fb76e1610d8af4c07. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:55.838212 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 2ec347e754ab4b7282e68b15a9bbf993 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ec347e754ab4b7282e68b15a9bbf993 already in progress: creating tablet
I20260505 07:53:55.839090 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 909306e1e5f34b7c8fbc0d86cafce531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.839643 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 3d6494590052459485059f5dce46f34f
I20260505 07:53:55.839705 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e707143a4e64725aa66db05b24999a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.840173 596 catalog_manager.cc:6741] ReplaceTablet: tablet 3d6494590052459485059f5dce46f34f deleted and replaced by tablet c822fafe72544ca99fde910c8e764755
I20260505 07:53:55.843711 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d6494590052459485059f5dce46f34f with delete_type TABLET_DATA_DELETED (replaced by tablet c822fafe72544ca99fde910c8e764755) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.844591 1171 ts_tablet_manager.cc:1929] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.844583 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d6494590052459485059f5dce46f34f with delete_type TABLET_DATA_DELETED (replaced by tablet c822fafe72544ca99fde910c8e764755) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.844722 1171 log.cc:1199] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/2ec347e754ab4b7282e68b15a9bbf993
W20260505 07:53:55.844937 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 956b5e05bda648349ba26d5c1ea3b279 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 956b5e05bda648349ba26d5c1ea3b279 already in progress: creating tablet
I20260505 07:53:55.845038 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 370be7a513ee49028c8d0662b43c8d91 (table default.Table_2a2802255e0743c18185473d0dcded48 [id=b2adaf7422c7403c8b331c0d33f0ad53]) successfully deleted
I20260505 07:53:55.845142 1171 ts_tablet_manager.cc:1950] T 2ec347e754ab4b7282e68b15a9bbf993 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.845191 1273 raft_consensus.cc:493] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:53:55.845588 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b0cf443e3044e56a26b0a8a61c9cf7d already in progress: creating tablet
W20260505 07:53:55.845676 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 2ec347e754ab4b7282e68b15a9bbf993 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2ec347e754ab4b7282e68b15a9bbf993 already in progress: creating tablet
W20260505 07:53:55.846330 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 956b5e05bda648349ba26d5c1ea3b279 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 956b5e05bda648349ba26d5c1ea3b279 already in progress: creating tablet
I20260505 07:53:55.846418 1171 tablet_replica.cc:333] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
W20260505 07:53:55.846463 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 03f31efbc98c484c950c70d5e84d2136 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 03f31efbc98c484c950c70d5e84d2136 already in progress: creating tablet
I20260505 07:53:55.846338 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 2ec347e754ab4b7282e68b15a9bbf993 (table default.Table_4d8c9530e5904e54ac3bdb2a82083180 [id=7e62bbf111c94a01b324b800bd6ced0b]) successfully deleted
I20260505 07:53:55.846619 1171 raft_consensus.cc:2243] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260505 07:53:55.846716 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 56cbbb89f6b84407b2c344ec62402b03 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56cbbb89f6b84407b2c344ec62402b03 already in progress: creating tablet
I20260505 07:53:55.847709 1175 tablet_replica.cc:333] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.847793 1171 raft_consensus.cc:2272] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.847893 1175 raft_consensus.cc:2243] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.848533 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d6494590052459485059f5dce46f34f with delete_type TABLET_DATA_DELETED (replaced by tablet c822fafe72544ca99fde910c8e764755) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.848729 1175 raft_consensus.cc:2272] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.848866 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ba04ccad67434c7884329b23efea4bfd" } modify_external_catalogs: true
I20260505 07:53:55.849040 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ba04ccad67434c7884329b23efea4bfd" } modify_external_catalogs: true
I20260505 07:53:55.849972 1273 raft_consensus.cc:515] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.850284 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 93d878ae0ddb42319560f74e0a490321 (DEFAULT_TABLE table=default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.851348 1273 leader_election.cc:290] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:55.851377 1171 ts_tablet_manager.cc:1916] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.851848 759 tablet_service.cc:1511] Processing CreateTablet for tablet 93d878ae0ddb42319560f74e0a490321 (DEFAULT_TABLE table=default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.852576 1175 ts_tablet_manager.cc:1916] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:55.852612 718 leader_election.cc:343] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8713b273c7b8438d8749f595ee5f1629
I20260505 07:53:55.852586 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93d878ae0ddb42319560f74e0a490321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.852703 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb" } modify_external_catalogs: true
I20260505 07:53:55.852876 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb" } modify_external_catalogs: true
I20260505 07:53:55.854643 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 8a60c015136246a3b02f70254c0bb67f
I20260505 07:53:55.856346 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a60c015136246a3b02f70254c0bb67f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.856767 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b8745d36d43a4f42a5b3d43a37c21e55" } modify_external_catalogs: true
I20260505 07:53:55.856945 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b8745d36d43a4f42a5b3d43a37c21e55" } modify_external_catalogs: true
I20260505 07:53:55.856791 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_32db8869c4e34bdf93b2c99c25d23805"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.858572 755 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a60c015136246a3b02f70254c0bb67f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:55.858750 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_32db8869c4e34bdf93b2c99c25d23805 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.859120 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet c6dcbebddcdc4ad9a788c01134e1101c
I20260505 07:53:55.859901 898 tablet_service.cc:1558] Processing DeleteTablet for tablet c6dcbebddcdc4ad9a788c01134e1101c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.860468 1176 tablet_bootstrap.cc:492] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:55.860616 1176 ts_tablet_manager.cc:1403] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.078s user 0.007s sys 0.000s
I20260505 07:53:55.860608 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_cb7435665bdd45ff8f517372a18d10b6" } modify_external_catalogs: true
I20260505 07:53:55.860765 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_cb7435665bdd45ff8f517372a18d10b6" } modify_external_catalogs: true
I20260505 07:53:55.860944 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_0bc65d0acfcb4ab2a829de8f1d831f9c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.861429 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0bc65d0acfcb4ab2a829de8f1d831f9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:55.862694 717 leader_election.cc:343] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 8713b273c7b8438d8749f595ee5f1629
I20260505 07:53:55.862840 717 leader_election.cc:304] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
W20260505 07:53:55.863690 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 03f31efbc98c484c950c70d5e84d2136 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 03f31efbc98c484c950c70d5e84d2136 already in progress: creating tablet
W20260505 07:53:55.863933 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b0cf443e3044e56a26b0a8a61c9cf7d already in progress: creating tablet
I20260505 07:53:55.863993 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93d878ae0ddb42319560f74e0a490321. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:55.864127 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 56cbbb89f6b84407b2c344ec62402b03 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 56cbbb89f6b84407b2c344ec62402b03 already in progress: creating tablet
I20260505 07:53:55.864907 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet c6dcbebddcdc4ad9a788c01134e1101c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.869110 755 tablet_service.cc:1558] Processing DeleteTablet for tablet c6dcbebddcdc4ad9a788c01134e1101c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.871575 1171 ts_tablet_manager.cc:1929] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.871737 1171 log.cc:1199] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/956b5e05bda648349ba26d5c1ea3b279
I20260505 07:53:55.872118 1171 ts_tablet_manager.cc:1950] T 956b5e05bda648349ba26d5c1ea3b279 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.873401 601 master_service.cc:954] ReplaceTablet: received request to replace tablet 32c008bd99224a7595cecc607e5496a9 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:55.873844 1171 tablet_replica.cc:333] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.874078 1171 raft_consensus.cc:2243] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.874236 1171 raft_consensus.cc:2272] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.871462 1273 raft_consensus.cc:2749] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:55.874696 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 956b5e05bda648349ba26d5c1ea3b279 (table default.Table_017976a3a6af42c9a9fc722661286c0e [id=d524458d873a48cb980e2f85490232d6]) successfully deleted
I20260505 07:53:55.874909 1171 ts_tablet_manager.cc:1916] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:55.874909 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b0cf443e3044e56a26b0a8a61c9cf7d already in progress: creating tablet
W20260505 07:53:55.875149 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 03f31efbc98c484c950c70d5e84d2136 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 03f31efbc98c484c950c70d5e84d2136 already in progress: creating tablet
I20260505 07:53:55.875527 1176 raft_consensus.cc:359] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.875710 1176 raft_consensus.cc:385] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.875834 1176 raft_consensus.cc:740] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.876215 1176 consensus_queue.cc:260] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.876670 1176 ts_tablet_manager.cc:1434] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260505 07:53:55.876991 1176 tablet_bootstrap.cc:492] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:55.878160 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet e85b44337f62494c997c0008fc492aab
I20260505 07:53:55.879182 1176 tablet_bootstrap.cc:654] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:55.879937 755 tablet_service.cc:1558] Processing DeleteTablet for tablet e85b44337f62494c997c0008fc492aab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.880524 898 tablet_service.cc:1558] Processing DeleteTablet for tablet e85b44337f62494c997c0008fc492aab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.880704 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 6429919aba17447dbdb393176360bbb0
I20260505 07:53:55.881515 755 tablet_service.cc:1558] Processing DeleteTablet for tablet 6429919aba17447dbdb393176360bbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.882788 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 6429919aba17447dbdb393176360bbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.882918 597 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e]). Expected version 2 got 0
I20260505 07:53:55.885039 1171 ts_tablet_manager.cc:1929] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.885238 1171 log.cc:1199] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/56cbbb89f6b84407b2c344ec62402b03
I20260505 07:53:55.885735 1171 ts_tablet_manager.cc:1950] T 56cbbb89f6b84407b2c344ec62402b03 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.886570 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 56cbbb89f6b84407b2c344ec62402b03 (table default.Table_da224a28c7e8451a8f4c04fcd2160c6b [id=bd575286f4e94df3b335b5e79d8f971c]) successfully deleted
I20260505 07:53:55.886719 1171 tablet_replica.cc:333] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.886946 1171 raft_consensus.cc:2243] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.887486 1171 raft_consensus.cc:2272] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.888195 1171 ts_tablet_manager.cc:1916] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:55.891433 1175 ts_tablet_manager.cc:1929] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.891551 1175 log.cc:1199] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/edb76b7961844443a9b41353c2d77d5a
I20260505 07:53:55.891945 1175 ts_tablet_manager.cc:1950] T edb76b7961844443a9b41353c2d77d5a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:55.892843 601 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 32c008bd99224a7595cecc607e5496a9
I20260505 07:53:55.892870 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet edb76b7961844443a9b41353c2d77d5a (table default.Table_dda9e1fe74264c2eab25cecc65079c93 [id=50e792c4b9044685ab0605ef0a9474a5]) successfully deleted
I20260505 07:53:55.893412 601 catalog_manager.cc:6741] ReplaceTablet: tablet 32c008bd99224a7595cecc607e5496a9 deleted and replaced by tablet a9494c3b6a99487596a030856b371b68
I20260505 07:53:55.894053 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 32c008bd99224a7595cecc607e5496a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a9494c3b6a99487596a030856b371b68) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.894253 755 tablet_service.cc:1558] Processing DeleteTablet for tablet 32c008bd99224a7595cecc607e5496a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a9494c3b6a99487596a030856b371b68) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.895766 1171 ts_tablet_manager.cc:1929] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:55.895931 1171 log.cc:1199] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/8573f8bd424641438e2aa6ff6869180e
I20260505 07:53:55.896334 1171 ts_tablet_manager.cc:1950] T 8573f8bd424641438e2aa6ff6869180e P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:55.896409 597 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e]). Expected version 2 got 0
I20260505 07:53:55.897348 1171 tablet_replica.cc:333] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:55.897322 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 8573f8bd424641438e2aa6ff6869180e (table default.Table_7d1be5262b41414f8a0fc37fba1cc28a [id=128f12d4ca8c43f6887ae3240610ef60]) successfully deleted
I20260505 07:53:55.897616 1171 raft_consensus.cc:2243] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:55.897821 1171 raft_consensus.cc:2272] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:55.898711 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 976359601ad74b149696dc7f555b550c
I20260505 07:53:55.899113 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 6429919aba17447dbdb393176360bbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.899705 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 976359601ad74b149696dc7f555b550c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:55.900072 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 976359601ad74b149696dc7f555b550c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:55.900679 898 tablet_service.cc:1511] Processing CreateTablet for tablet 93d878ae0ddb42319560f74e0a490321 (DEFAULT_TABLE table=default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.901116 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93d878ae0ddb42319560f74e0a490321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.901319 752 tablet_service.cc:1511] Processing CreateTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 (DEFAULT_TABLE table=default.Table_8cdf9d3d3e6349f4b8d3536f78451100 [id=2c91868ddba84103b9a5c78201842a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.901650 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef4f516e64a44fd8cd33674fb808067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.902668 759 tablet_service.cc:1511] Processing CreateTablet for tablet 4662d4a3d88d4c2883a35048d6e11eb9 (DEFAULT_TABLE table=default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9 [id=3aa673bdfdb2432bb2174d4a520d7a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.903153 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4662d4a3d88d4c2883a35048d6e11eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.904109 890 tablet_service.cc:1511] Processing CreateTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 (DEFAULT_TABLE table=default.Table_8cdf9d3d3e6349f4b8d3536f78451100 [id=2c91868ddba84103b9a5c78201842a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.904516 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef4f516e64a44fd8cd33674fb808067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.905774 1029 tablet_service.cc:1558] Processing DeleteTablet for tablet 32c008bd99224a7595cecc607e5496a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a9494c3b6a99487596a030856b371b68) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.909030 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet e85b44337f62494c997c0008fc492aab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.909511 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 976359601ad74b149696dc7f555b550c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.910297 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a60c015136246a3b02f70254c0bb67f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:55.912518 1029 tablet_service.cc:1511] Processing CreateTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 (DEFAULT_TABLE table=default.Table_8cdf9d3d3e6349f4b8d3536f78451100 [id=2c91868ddba84103b9a5c78201842a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.912673 755 tablet_service.cc:1511] Processing CreateTablet for tablet 85d485ae2d1d4d76a6a5d988cec8291a (DEFAULT_TABLE table=default.Table_9912957f51d44c0ab0b2aaaf516ad36f [id=6e71d3f928fc45c191f64a945980a107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.913041 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d485ae2d1d4d76a6a5d988cec8291a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.913275 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ef4f516e64a44fd8cd33674fb808067. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.913344 897 tablet_service.cc:1511] Processing CreateTablet for tablet 85d485ae2d1d4d76a6a5d988cec8291a (DEFAULT_TABLE table=default.Table_9912957f51d44c0ab0b2aaaf516ad36f [id=6e71d3f928fc45c191f64a945980a107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.913671 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d485ae2d1d4d76a6a5d988cec8291a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.915863 750 tablet_service.cc:1511] Processing CreateTablet for tablet bf76e868bbcb44b0b1887c84dd27f093 (DEFAULT_TABLE table=default.Table_7c3d336ba1db41b9b26a2fe5475bbab0 [id=50b5f277bb144f4893d158d179bd0fb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.916245 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf76e868bbcb44b0b1887c84dd27f093. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.916533 754 tablet_service.cc:1511] Processing CreateTablet for tablet e278b86e7b5d444ebad8d0f6921d4c62 (DEFAULT_TABLE table=default.Table_d0fa020605d347b3ad7096aed39c552c [id=d53c653124364ea5acc357977c9db864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.916873 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e278b86e7b5d444ebad8d0f6921d4c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.919538 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 85d485ae2d1d4d76a6a5d988cec8291a (DEFAULT_TABLE table=default.Table_9912957f51d44c0ab0b2aaaf516ad36f [id=6e71d3f928fc45c191f64a945980a107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.920336 756 tablet_service.cc:1511] Processing CreateTablet for tablet 840eebf9e7404bf7807b7e51996957e5 (DEFAULT_TABLE table=default.Table_5d02989a34e8412b819674082ae217f7 [id=eb677b61f4ea498aaf962c74b7c7d5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.920481 896 tablet_service.cc:1511] Processing CreateTablet for tablet bf76e868bbcb44b0b1887c84dd27f093 (DEFAULT_TABLE table=default.Table_7c3d336ba1db41b9b26a2fe5475bbab0 [id=50b5f277bb144f4893d158d179bd0fb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.920892 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf76e868bbcb44b0b1887c84dd27f093. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.920951 761 tablet_service.cc:1511] Processing CreateTablet for tablet 9044763bed1a41739ecce4def4cca1fe (DEFAULT_TABLE table=default.Table_da224a28c7e8451a8f4c04fcd2160c6b [id=bd575286f4e94df3b335b5e79d8f971c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.921706 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840eebf9e7404bf7807b7e51996957e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.921875 759 tablet_service.cc:1511] Processing CreateTablet for tablet f1058fcb9fa6457a85489049ca1d9ada (DEFAULT_TABLE table=default.Table_d27b912f6fe14120afb18354e772a658 [id=42ae40c0861e4489b724c9b8e3b590d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.923290 899 tablet_service.cc:1511] Processing CreateTablet for tablet 840eebf9e7404bf7807b7e51996957e5 (DEFAULT_TABLE table=default.Table_5d02989a34e8412b819674082ae217f7 [id=eb677b61f4ea498aaf962c74b7c7d5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.923353 900 tablet_service.cc:1511] Processing CreateTablet for tablet f1058fcb9fa6457a85489049ca1d9ada (DEFAULT_TABLE table=default.Table_d27b912f6fe14120afb18354e772a658 [id=42ae40c0861e4489b724c9b8e3b590d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.923650 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840eebf9e7404bf7807b7e51996957e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.925594 893 tablet_service.cc:1511] Processing CreateTablet for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (DEFAULT_TABLE table=default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.926028 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d1ce00e487d4702a6affcad57a7fbe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.926429 895 tablet_service.cc:1511] Processing CreateTablet for tablet 9044763bed1a41739ecce4def4cca1fe (DEFAULT_TABLE table=default.Table_da224a28c7e8451a8f4c04fcd2160c6b [id=bd575286f4e94df3b335b5e79d8f971c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.926709 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9044763bed1a41739ecce4def4cca1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.927373 901 tablet_service.cc:1511] Processing CreateTablet for tablet e278b86e7b5d444ebad8d0f6921d4c62 (DEFAULT_TABLE table=default.Table_d0fa020605d347b3ad7096aed39c552c [id=d53c653124364ea5acc357977c9db864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.913658 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 4662d4a3d88d4c2883a35048d6e11eb9 (DEFAULT_TABLE table=default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9 [id=3aa673bdfdb2432bb2174d4a520d7a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.929282 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1058fcb9fa6457a85489049ca1d9ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.928403 892 tablet_service.cc:1511] Processing CreateTablet for tablet 323269e708964848a94c64e6a781a860 (DEFAULT_TABLE table=default.Table_55630f6d78594e3d810e3291317b348c [id=09bd9815349347ab9b75c4242c354c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.931022 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e278b86e7b5d444ebad8d0f6921d4c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.931596 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 9044763bed1a41739ecce4def4cca1fe (DEFAULT_TABLE table=default.Table_da224a28c7e8451a8f4c04fcd2160c6b [id=bd575286f4e94df3b335b5e79d8f971c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.932145 1175 tablet_replica.cc:333] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:55.932667 894 tablet_service.cc:1511] Processing CreateTablet for tablet 78a6470720984761a8606abeafbef992 (DEFAULT_TABLE table=default.Table_5cd903011cdc44a392680507ed9088c8 [id=a3c321e46ecd4b07809a9c663305afb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.932817 891 tablet_service.cc:1511] Processing CreateTablet for tablet accb2dedc30e494ebd22c33594cee661 (DEFAULT_TABLE table=default.Table_042d1a9abf8948aebac31e5f4f810019 [id=860ab29ce1a44ad1b2480dbdb2b40000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.933046 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (DEFAULT_TABLE table=default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.933802 1021 tablet_service.cc:1511] Processing CreateTablet for tablet f1058fcb9fa6457a85489049ca1d9ada (DEFAULT_TABLE table=default.Table_d27b912f6fe14120afb18354e772a658 [id=42ae40c0861e4489b724c9b8e3b590d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.933979 1022 tablet_service.cc:1511] Processing CreateTablet for tablet accb2dedc30e494ebd22c33594cee661 (DEFAULT_TABLE table=default.Table_042d1a9abf8948aebac31e5f4f810019 [id=860ab29ce1a44ad1b2480dbdb2b40000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.934598 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 78a6470720984761a8606abeafbef992 (DEFAULT_TABLE table=default.Table_5cd903011cdc44a392680507ed9088c8 [id=a3c321e46ecd4b07809a9c663305afb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.936028 1025 tablet_service.cc:1511] Processing CreateTablet for tablet e278b86e7b5d444ebad8d0f6921d4c62 (DEFAULT_TABLE table=default.Table_d0fa020605d347b3ad7096aed39c552c [id=d53c653124364ea5acc357977c9db864]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.937054 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 323269e708964848a94c64e6a781a860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.938740 1024 tablet_service.cc:1511] Processing CreateTablet for tablet bf76e868bbcb44b0b1887c84dd27f093 (DEFAULT_TABLE table=default.Table_7c3d336ba1db41b9b26a2fe5475bbab0 [id=50b5f277bb144f4893d158d179bd0fb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.939025 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d485ae2d1d4d76a6a5d988cec8291a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.940552 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet accb2dedc30e494ebd22c33594cee661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.940534 889 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 (DEFAULT_TABLE table=default.Table_1da2218f525749af888a9da48524bc11 [id=1dc1ab0d9aa44adba092a585e5b98640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.941265 1172 tablet_bootstrap.cc:492] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:55.946878 1172 ts_tablet_manager.cc:1403] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.132s user 0.049s sys 0.013s
I20260505 07:53:55.947327 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78a6470720984761a8606abeafbef992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.947880 890 tablet_service.cc:1511] Processing CreateTablet for tablet 4662d4a3d88d4c2883a35048d6e11eb9 (DEFAULT_TABLE table=default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9 [id=3aa673bdfdb2432bb2174d4a520d7a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.948134 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca2852cee84476dbe5e5072a43087a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.941618 888 tablet_service.cc:1511] Processing CreateTablet for tablet 17327cee69a24e0cb493ff55f948bf6c (DEFAULT_TABLE table=default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.949246 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 840eebf9e7404bf7807b7e51996957e5 (DEFAULT_TABLE table=default.Table_5d02989a34e8412b819674082ae217f7 [id=eb677b61f4ea498aaf962c74b7c7d5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.949950 1178 tablet_replica.cc:333] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:55.948717 887 tablet_service.cc:1511] Processing CreateTablet for tablet 9a06f77255544d7dabb71a4f70ada809 (DEFAULT_TABLE table=default.Table_4b3acc65322b4dfe93c4f1d049b573e5 [id=2a8cda02af654aa5b0c161e04968c735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.952530 758 tablet_service.cc:1511] Processing CreateTablet for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (DEFAULT_TABLE table=default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.953140 751 tablet_service.cc:1511] Processing CreateTablet for tablet accb2dedc30e494ebd22c33594cee661 (DEFAULT_TABLE table=default.Table_042d1a9abf8948aebac31e5f4f810019 [id=860ab29ce1a44ad1b2480dbdb2b40000]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.953480 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1058fcb9fa6457a85489049ca1d9ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.953784 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9044763bed1a41739ecce4def4cca1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.954816 762 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 (DEFAULT_TABLE table=default.Table_1da2218f525749af888a9da48524bc11 [id=1dc1ab0d9aa44adba092a585e5b98640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.955018 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17327cee69a24e0cb493ff55f948bf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.955159 757 tablet_service.cc:1511] Processing CreateTablet for tablet 323269e708964848a94c64e6a781a860 (DEFAULT_TABLE table=default.Table_55630f6d78594e3d810e3291317b348c [id=09bd9815349347ab9b75c4242c354c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.955484 753 tablet_service.cc:1511] Processing CreateTablet for tablet 17327cee69a24e0cb493ff55f948bf6c (DEFAULT_TABLE table=default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.956084 1172 raft_consensus.cc:359] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.956305 1172 raft_consensus.cc:385] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:55.956385 1172 raft_consensus.cc:740] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:55.959703 596 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8fd577cca7804ca198558aa2510e53e6" } new_table_name: "default.Table_0562bf620b5f46ecab060957ac1ca8bf" modify_external_catalogs: true
I20260505 07:53:55.961198 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a06f77255544d7dabb71a4f70ada809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.959775 1172 consensus_queue.cc:260] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:55.966143 1172 ts_tablet_manager.cc:1434] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
W20260505 07:53:55.967434 596 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 23fa401bb4634022bca3c6b23421e4d5 consensus state has no leader
I20260505 07:53:55.969708 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_e23a9873d96a4ca8ba1bbe8c4976bc59" } new_table_name: "default.Table_7f72af6be89b4942a09acb0dc9615952" modify_external_catalogs: true
I20260505 07:53:55.968871 1019 tablet_service.cc:1511] Processing CreateTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 (DEFAULT_TABLE table=default.Table_1da2218f525749af888a9da48524bc11 [id=1dc1ab0d9aa44adba092a585e5b98640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.969340 1020 tablet_service.cc:1511] Processing CreateTablet for tablet 323269e708964848a94c64e6a781a860 (DEFAULT_TABLE table=default.Table_55630f6d78594e3d810e3291317b348c [id=09bd9815349347ab9b75c4242c354c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.969135 1018 tablet_service.cc:1511] Processing CreateTablet for tablet 17327cee69a24e0cb493ff55f948bf6c (DEFAULT_TABLE table=default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.969730 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 9a06f77255544d7dabb71a4f70ada809 (DEFAULT_TABLE table=default.Table_4b3acc65322b4dfe93c4f1d049b573e5 [id=2a8cda02af654aa5b0c161e04968c735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.972337 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_287f96f2dc994ce1b71c17af88b85b63"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.972862 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_287f96f2dc994ce1b71c17af88b85b63 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:55.976501 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf consensus state has no leader
I20260505 07:53:55.980882 1033 tablet_service.cc:1511] Processing CreateTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e (DEFAULT_TABLE table=default.Table_1e4ca2caf1834843a4b8fc347fc080bc [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.982427 899 tablet_service.cc:1511] Processing CreateTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e (DEFAULT_TABLE table=default.Table_1e4ca2caf1834843a4b8fc347fc080bc [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.982963 760 tablet_service.cc:1511] Processing CreateTablet for tablet 9a06f77255544d7dabb71a4f70ada809 (DEFAULT_TABLE table=default.Table_4b3acc65322b4dfe93c4f1d049b573e5 [id=2a8cda02af654aa5b0c161e04968c735]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.983626 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1b644b185b194940a8a8a90595ae9e90"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.983861 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_5edc57ff557343629eb69ed0fe818ded"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.984089 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b644b185b194940a8a8a90595ae9e90 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:55.984261 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5edc57ff557343629eb69ed0fe818ded in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.985343 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_0654ae2daf60435697eeca15ba0c19d7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.985785 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0654ae2daf60435697eeca15ba0c19d7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.985872 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_32db8869c4e34bdf93b2c99c25d23805" } new_table_name: "default.Table_613832387c45465197e11f83984f0304" modify_external_catalogs: true
I20260505 07:53:55.987751 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca2852cee84476dbe5e5072a43087a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.988353 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d1ce00e487d4702a6affcad57a7fbe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.988628 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840eebf9e7404bf7807b7e51996957e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.988884 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1058fcb9fa6457a85489049ca1d9ada. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.989169 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17327cee69a24e0cb493ff55f948bf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.989403 896 tablet_service.cc:1511] Processing CreateTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 (DEFAULT_TABLE table=default.Table_32db8869c4e34bdf93b2c99c25d23805 [id=e59beab126684f548325a29eb663177b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.989517 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78a6470720984761a8606abeafbef992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.989892 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4662d4a3d88d4c2883a35048d6e11eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.990309 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_01a5eabcd2ff41fca0d5c94e26775754"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:55.990769 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_01a5eabcd2ff41fca0d5c94e26775754 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.988715 1016 tablet_service.cc:1511] Processing CreateTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 (DEFAULT_TABLE table=default.Table_32db8869c4e34bdf93b2c99c25d23805 [id=e59beab126684f548325a29eb663177b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:55.992867 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e278b86e7b5d444ebad8d0f6921d4c62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.994658 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4783dd0b2fbe40219852d850fc0d9a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:55.995976 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_0ec687e90aea4825b28b36eda22678fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:55.996342 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ca2852cee84476dbe5e5072a43087a1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:55.996379 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ec687e90aea4825b28b36eda22678fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:55.997839 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a06f77255544d7dabb71a4f70ada809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.000257 886 tablet_service.cc:1511] Processing CreateTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 (DEFAULT_TABLE table=default.Table_344c494090d44fa8b4102b525e7ca099 [id=d8050f3674fe49f7a8af288e6ea57a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.000972 885 tablet_service.cc:1511] Processing CreateTablet for tablet 397d33e908af4a0bb8dde26d273be01a (DEFAULT_TABLE table=default.Table_499d356daab040e29ca2f5eb2941d790 [id=bf9eb924eb1e4586a6346bcf43be9a30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.001324 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 323269e708964848a94c64e6a781a860. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:56.003660 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4783dd0b2fbe40219852d850fc0d9a82 consensus state has no leader
I20260505 07:53:56.004218 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9044763bed1a41739ecce4def4cca1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.004556 884 tablet_service.cc:1511] Processing CreateTablet for tablet a9494c3b6a99487596a030856b371b68 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.004822 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet accb2dedc30e494ebd22c33594cee661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.006172 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4662d4a3d88d4c2883a35048d6e11eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.007881 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_287f96f2dc994ce1b71c17af88b85b63" } new_table_name: "default.Table_577ff393d2af447a915f4460ab7692f1" modify_external_catalogs: true
I20260505 07:53:56.007993 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_eb5ee86732cf4963809c3838e7654571"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.008421 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb5ee86732cf4963809c3838e7654571 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.010103 596 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_344daeac12fd48cfab632d6a4f90ba30" } new_table_name: "default.Table_bd073edd5f1c4a70909afeb86fd7fd11" modify_external_catalogs: true
I20260505 07:53:56.010150 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f8f7a899e8c94e75958bd11fed643bc6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.010553 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f8f7a899e8c94e75958bd11fed643bc6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.015379 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 (DEFAULT_TABLE table=default.Table_344c494090d44fa8b4102b525e7ca099 [id=d8050f3674fe49f7a8af288e6ea57a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.016852 883 tablet_service.cc:1511] Processing CreateTablet for tablet 70404184502a4394a737518220f2124f (DEFAULT_TABLE table=default.Table_cf3b3e8080ba449a9f92b5175e2193b1 [id=22feb370028a4bd59568731eaa7b090c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.017362 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6da10815e0dc41a1a07b6786882fffd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.020284 882 tablet_service.cc:1511] Processing CreateTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 (DEFAULT_TABLE table=default.Table_0562bf620b5f46ecab060957ac1ca8bf [id=f47382441b814a62b5ff875ae632b915]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.021670 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 70404184502a4394a737518220f2124f (DEFAULT_TABLE table=default.Table_cf3b3e8080ba449a9f92b5175e2193b1 [id=22feb370028a4bd59568731eaa7b090c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.022255 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70404184502a4394a737518220f2124f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.030645 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cac5064f68d24ebe95b5a0b8e10f463e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.031311 1029 tablet_service.cc:1511] Processing CreateTablet for tablet 397d33e908af4a0bb8dde26d273be01a (DEFAULT_TABLE table=default.Table_499d356daab040e29ca2f5eb2941d790 [id=bf9eb924eb1e4586a6346bcf43be9a30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.031806 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 397d33e908af4a0bb8dde26d273be01a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:56.047721 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 06904a54b1c5463da31af0bc00fb606f has no consensus state
I20260505 07:53:56.053086 756 tablet_service.cc:1511] Processing CreateTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e (DEFAULT_TABLE table=default.Table_1e4ca2caf1834843a4b8fc347fc080bc [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.053475 1171 ts_tablet_manager.cc:1916] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:56.054821 1175 raft_consensus.cc:2243] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:56.054984 1175 raft_consensus.cc:2272] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:56.055051 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a06f77255544d7dabb71a4f70ada809. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.054533 755 tablet_service.cc:1511] Processing CreateTablet for tablet 78a6470720984761a8606abeafbef992 (DEFAULT_TABLE table=default.Table_5cd903011cdc44a392680507ed9088c8 [id=a3c321e46ecd4b07809a9c663305afb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.057139 1172 tablet_bootstrap.cc:492] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:56.057261 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 03f31efbc98c484c950c70d5e84d2136 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:56.058677 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78a6470720984761a8606abeafbef992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.058851 1172 tablet_bootstrap.cc:654] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.054432 749 tablet_service.cc:1511] Processing CreateTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 (DEFAULT_TABLE table=default.Table_32db8869c4e34bdf93b2c99c25d23805 [id=e59beab126684f548325a29eb663177b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.059499 747 tablet_service.cc:1511] Processing CreateTablet for tablet a9494c3b6a99487596a030856b371b68 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.059962 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cac5064f68d24ebe95b5a0b8e10f463e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.060474 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cac5064f68d24ebe95b5a0b8e10f463e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.062597 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_efeac090fb094bdf82f547af366d562c" } modify_external_catalogs: true
I20260505 07:53:56.062803 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_efeac090fb094bdf82f547af366d562c" } modify_external_catalogs: true
I20260505 07:53:56.064397 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4783dd0b2fbe40219852d850fc0d9a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.065567 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_344c494090d44fa8b4102b525e7ca099" } modify_external_catalogs: true
I20260505 07:53:56.065747 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_344c494090d44fa8b4102b525e7ca099" } modify_external_catalogs: true
I20260505 07:53:56.066457 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 (DEFAULT_TABLE table=default.Table_0562bf620b5f46ecab060957ac1ca8bf [id=f47382441b814a62b5ff875ae632b915]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.067131 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf76e868bbcb44b0b1887c84dd27f093. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.067785 597 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e1a9420a81844d2092381123d9297661 (table default.Table_422080d5c75f422fb88c26b76ffc61c4 [id=0d8de70d77c54c2491b257d09c8474c8]). Expected version 2 got 0
I20260505 07:53:56.068781 1178 raft_consensus.cc:2243] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:56.071646 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23fa401bb4634022bca3c6b23421e4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.073836 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6da10815e0dc41a1a07b6786882fffd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.074426 1178 raft_consensus.cc:2272] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:56.074986 1175 ts_tablet_manager.cc:1916] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:56.075309 750 tablet_service.cc:1511] Processing CreateTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 (DEFAULT_TABLE table=default.Table_344c494090d44fa8b4102b525e7ca099 [id=d8050f3674fe49f7a8af288e6ea57a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.037669 1023 tablet_service.cc:1511] Processing CreateTablet for tablet a9494c3b6a99487596a030856b371b68 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.081131 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_27c52b9b060d4dd5a80799a987d5b7c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.081727 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27c52b9b060d4dd5a80799a987d5b7c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:56.081867 596 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 067c32ecccb44bca8bf9cc3f6c229326 consensus state has no leader
I20260505 07:53:56.082777 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b31d40ee188f46d0bdd481aa88730544" } modify_external_catalogs: true
I20260505 07:53:56.083961 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_bd283510c1f14a9fb29a9870d33c9971" } modify_external_catalogs: true
I20260505 07:53:56.084138 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_bd283510c1f14a9fb29a9870d33c9971" } modify_external_catalogs: true
I20260505 07:53:56.083348 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b31d40ee188f46d0bdd481aa88730544" } modify_external_catalogs: true
I20260505 07:53:56.089154 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_063cc9f08ab54458b9e277ca65091b11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.089702 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_063cc9f08ab54458b9e277ca65091b11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.094930 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet ccdde3e5a13e436d9f2328831e155adb
I20260505 07:53:56.096477 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 6da10815e0dc41a1a07b6786882fffd3
I20260505 07:53:56.099036 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 397d33e908af4a0bb8dde26d273be01a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.101224 901 tablet_service.cc:1511] Processing CreateTablet for tablet 003cd4472b424aacbba8c5f89900e228 (DEFAULT_TABLE table=default.Table_bd283510c1f14a9fb29a9870d33c9971 [id=ae40934b942a4e0e8ac882d9dab6686f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.101218 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_5cd903011cdc44a392680507ed9088c8" } modify_external_catalogs: true
I20260505 07:53:56.101890 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_5cd903011cdc44a392680507ed9088c8" } modify_external_catalogs: true
I20260505 07:53:56.105274 746 tablet_service.cc:1511] Processing CreateTablet for tablet 70404184502a4394a737518220f2124f (DEFAULT_TABLE table=default.Table_cf3b3e8080ba449a9f92b5175e2193b1 [id=22feb370028a4bd59568731eaa7b090c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.106818 754 tablet_service.cc:1511] Processing CreateTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 (DEFAULT_TABLE table=default.Table_0562bf620b5f46ecab060957ac1ca8bf [id=f47382441b814a62b5ff875ae632b915]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.108822 745 tablet_service.cc:1511] Processing CreateTablet for tablet c822fafe72544ca99fde910c8e764755 (DEFAULT_TABLE table=default.Table_3a027f114e1e4d5fabe7ddc8f584e3ec [id=85e91b69517e415194bc6c0dd62ccf9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.109912 748 tablet_service.cc:1511] Processing CreateTablet for tablet 397d33e908af4a0bb8dde26d273be01a (DEFAULT_TABLE table=default.Table_499d356daab040e29ca2f5eb2941d790 [id=bf9eb924eb1e4586a6346bcf43be9a30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.110653 744 tablet_service.cc:1511] Processing CreateTablet for tablet 003cd4472b424aacbba8c5f89900e228 (DEFAULT_TABLE table=default.Table_bd283510c1f14a9fb29a9870d33c9971 [id=ae40934b942a4e0e8ac882d9dab6686f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.110996 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4783dd0b2fbe40219852d850fc0d9a82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.111706 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6da10815e0dc41a1a07b6786882fffd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.111907 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23fa401bb4634022bca3c6b23421e4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.114348 743 tablet_service.cc:1511] Processing CreateTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 (DEFAULT_TABLE table=default.Table_344daeac12fd48cfab632d6a4f90ba30 [id=b1689bb0b68c4d21b1689c04b2904022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.120478 593 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e1a9420a81844d2092381123d9297661 (table default.Table_422080d5c75f422fb88c26b76ffc61c4 [id=0d8de70d77c54c2491b257d09c8474c8]). Expected version 2 got 0
I20260505 07:53:56.126886 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c822fafe72544ca99fde910c8e764755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.131881 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70404184502a4394a737518220f2124f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.134222 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70404184502a4394a737518220f2124f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.134060 893 tablet_service.cc:1511] Processing CreateTablet for tablet c822fafe72544ca99fde910c8e764755 (DEFAULT_TABLE table=default.Table_3a027f114e1e4d5fabe7ddc8f584e3ec [id=85e91b69517e415194bc6c0dd62ccf9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.135907 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 323269e708964848a94c64e6a781a860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.137243 891 tablet_service.cc:1511] Processing CreateTablet for tablet 83e62b7af9b847b7a9005e4345905c0d (DEFAULT_TABLE table=default.Table_0bc65d0acfcb4ab2a829de8f1d831f9c [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.138084 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.139473 897 tablet_service.cc:1558] Processing DeleteTablet for tablet ccdde3e5a13e436d9f2328831e155adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.140164 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 003cd4472b424aacbba8c5f89900e228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.148224 887 tablet_service.cc:1511] Processing CreateTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 (DEFAULT_TABLE table=default.Table_344daeac12fd48cfab632d6a4f90ba30 [id=b1689bb0b68c4d21b1689c04b2904022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.152093 752 tablet_service.cc:1558] Processing DeleteTablet for tablet ccdde3e5a13e436d9f2328831e155adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.155736 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9494c3b6a99487596a030856b371b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.155757 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet accb2dedc30e494ebd22c33594cee661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.158047 754 tablet_service.cc:1511] Processing CreateTablet for tablet 83e62b7af9b847b7a9005e4345905c0d (DEFAULT_TABLE table=default.Table_0bc65d0acfcb4ab2a829de8f1d831f9c [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.159655 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 003cd4472b424aacbba8c5f89900e228 (DEFAULT_TABLE table=default.Table_bd283510c1f14a9fb29a9870d33c9971 [id=ae40934b942a4e0e8ac882d9dab6686f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.161284 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 23fa401bb4634022bca3c6b23421e4d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.162003 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.166826 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17327cee69a24e0cb493ff55f948bf6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.167047 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83e62b7af9b847b7a9005e4345905c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.167303 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 003cd4472b424aacbba8c5f89900e228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.170302 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 397d33e908af4a0bb8dde26d273be01a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.170403 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c822fafe72544ca99fde910c8e764755. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.171852 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 067c32ecccb44bca8bf9cc3f6c229326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.173516 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_697f13aef35b49989ffebd1549f8e930" } new_table_name: "default.Table_4ddf6f25bf834287b5808ee1c64474bd" modify_external_catalogs: true
I20260505 07:53:56.174577 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83e62b7af9b847b7a9005e4345905c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.175231 1171 ts_tablet_manager.cc:1929] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:56.175360 1171 log.cc:1199] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/451e99a0dfec4823812e5991147b132e
I20260505 07:53:56.177834 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d1ce00e487d4702a6affcad57a7fbe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.178210 1171 ts_tablet_manager.cc:1950] T 451e99a0dfec4823812e5991147b132e P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:56.178236 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ff28fa06b61e4bcb9f00bfd8bcc0e447" } new_table_name: "default.Table_98b055003e064e57b2ec488952e58844" modify_external_catalogs: true
I20260505 07:53:56.178774 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 451e99a0dfec4823812e5991147b132e (table default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95]) successfully deleted
I20260505 07:53:56.179126 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 067c32ecccb44bca8bf9cc3f6c229326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.179397 1171 tablet_replica.cc:333] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:56.179620 1171 raft_consensus.cc:2243] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:56.179720 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9494c3b6a99487596a030856b371b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.179803 1171 raft_consensus.cc:2272] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:56.180506 1171 ts_tablet_manager.cc:1916] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:56.186875 1171 ts_tablet_manager.cc:1929] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:56.186981 1171 log.cc:1199] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/c7652191b1d44ba59d856a2b0d7017b1
I20260505 07:53:56.188208 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 1852f7f4ff594639908b03730b68fe86 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:56.195654 1172 tablet_bootstrap.cc:492] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:56.195752 1171 ts_tablet_manager.cc:1950] T c7652191b1d44ba59d856a2b0d7017b1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:56.195818 1172 ts_tablet_manager.cc:1403] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.139s user 0.008s sys 0.039s
I20260505 07:53:56.197053 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet c7652191b1d44ba59d856a2b0d7017b1 (table default.Table_6e9d636b8b314e5b888c725d34775180 [id=73c0bcbf3485405589f58b8175bfc3b9]) successfully deleted
I20260505 07:53:56.197937 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 1ef4f516e64a44fd8cd33674fb808067 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:56.200766 1172 raft_consensus.cc:359] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.200953 1172 raft_consensus.cc:385] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260505 07:53:56.200953 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 3d6494590052459485059f5dce46f34f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d6494590052459485059f5dce46f34f already in progress: creating tablet
W20260505 07:53:56.201177 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 8a60c015136246a3b02f70254c0bb67f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a60c015136246a3b02f70254c0bb67f already in progress: creating tablet
W20260505 07:53:56.201360 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet c6dcbebddcdc4ad9a788c01134e1101c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c6dcbebddcdc4ad9a788c01134e1101c already in progress: creating tablet
W20260505 07:53:56.201581 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e85b44337f62494c997c0008fc492aab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e85b44337f62494c997c0008fc492aab already in progress: creating tablet
I20260505 07:53:56.201682 1172 raft_consensus.cc:740] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:53:56.201794 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 6429919aba17447dbdb393176360bbb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6429919aba17447dbdb393176360bbb0 already in progress: creating tablet
W20260505 07:53:56.202226 587 consensus_peers.cc:597] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 -> Peer 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565): Couldn't send request to peer 23c57a2c927c4c54bccecfeb3a775311. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.61:33565: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260505 07:53:56.201932 1172 consensus_queue.cc:260] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.204519 1172 ts_tablet_manager.cc:1434] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260505 07:53:56.204849 1172 tablet_bootstrap.cc:492] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:56.205168 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_659ddda68e194bb7ae10409bbef074c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.205688 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_659ddda68e194bb7ae10409bbef074c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.206410 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f6734bb37ebf474982961491f124f0f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.206535 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 32c008bd99224a7595cecc607e5496a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32c008bd99224a7595cecc607e5496a9 already in progress: creating tablet
W20260505 07:53:56.206746 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 976359601ad74b149696dc7f555b550c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 976359601ad74b149696dc7f555b550c already in progress: creating tablet
W20260505 07:53:56.206967 588 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 6da10815e0dc41a1a07b6786882fffd3 because the tablet was not found. No further retry: Not found: Tablet not found: 6da10815e0dc41a1a07b6786882fffd3
W20260505 07:53:56.206894 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6734bb37ebf474982961491f124f0f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:56.207227 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet ccdde3e5a13e436d9f2328831e155adb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccdde3e5a13e436d9f2328831e155adb already in progress: creating tablet
I20260505 07:53:56.213783 1172 tablet_bootstrap.cc:654] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.215727 1229 raft_consensus.cc:493] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:56.215986 1229 raft_consensus.cc:515] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.216583 1229 leader_election.cc:290] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:56.217308 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a74b65d5d6459293ab4461f4d34cf2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:56.219048 854 leader_election.cc:343] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:56.222242 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 78a6470720984761a8606abeafbef992
I20260505 07:53:56.223733 599 master_service.cc:954] ReplaceTablet: received request to replace tablet 1838f89dc14a47f99f95f4cc5fdefd70 from {username='slave'} at 127.0.0.1:47238
W20260505 07:53:56.223894 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 909306e1e5f34b7c8fbc0d86cafce531 consensus state has no leader
W20260505 07:53:56.224148 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b404e50ae092419c9396772b3b14eeaa consensus state has no leader
I20260505 07:53:56.224243 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_bd073edd5f1c4a70909afeb86fd7fd11" } new_table_name: "default.Table_92bd525516eb48ff97bb8fc8c1c2b702" modify_external_catalogs: true
I20260505 07:53:56.224524 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_92db3a6ebf824942966e677a2cd23ce1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.224988 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_92db3a6ebf824942966e677a2cd23ce1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.226737 1178 ts_tablet_manager.cc:1916] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:56.227337 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 3d6494590052459485059f5dce46f34f from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:56.229251 1176 tablet_bootstrap.cc:492] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:56.229413 1176 ts_tablet_manager.cc:1403] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.352s user 0.026s sys 0.014s
I20260505 07:53:56.230222 1176 raft_consensus.cc:359] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.230413 1176 raft_consensus.cc:385] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.230480 1176 raft_consensus.cc:740] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:56.230706 1176 consensus_queue.cc:260] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.236395 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_82a545edb422426ab7eaeb4ae68a1a3f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.236953 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82a545edb422426ab7eaeb4ae68a1a3f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.239115 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 78a6470720984761a8606abeafbef992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.240352 1181 tablet_bootstrap.cc:492] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:56.240507 1181 ts_tablet_manager.cc:1403] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.485s user 0.013s sys 0.001s
I20260505 07:53:56.242806 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.243338 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.244803 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a74b65d5d6459293ab4461f4d34cf2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:56.245374 856 leader_election.cc:343] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:56.245553 856 leader_election.cc:304] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:56.245123 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 4a79578757824bfc9df129e3475e7af3
I20260505 07:53:56.247864 1229 raft_consensus.cc:2749] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:56.243893 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 (DEFAULT_TABLE table=default.Table_344daeac12fd48cfab632d6a4f90ba30 [id=b1689bb0b68c4d21b1689c04b2904022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.249783 893 tablet_service.cc:1558] Processing DeleteTablet for tablet 25a74b65d5d6459293ab4461f4d34cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.250010 1171 tablet_replica.cc:333] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
W20260505 07:53:56.249984 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 78a6470720984761a8606abeafbef992 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78a6470720984761a8606abeafbef992 already in progress: creating tablet
I20260505 07:53:56.250207 1171 raft_consensus.cc:2243] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:56.250360 1171 raft_consensus.cc:2272] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:56.251658 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0bc65d0acfcb4ab2a829de8f1d831f9c" } new_table_name: "default.Table_2071445d362745d9b2bbc7712d032c17" modify_external_catalogs: true
I20260505 07:53:56.252091 1171 ts_tablet_manager.cc:1916] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:56.252774 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a79578757824bfc9df129e3475e7af3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.255434 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_59819a75045240de818a627f6036f8e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.255957 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59819a75045240de818a627f6036f8e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:56.256112 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 067c32ecccb44bca8bf9cc3f6c229326 consensus state has no leader
I20260505 07:53:56.256999 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_085f78183e1e4000a83ccbb0ed350340"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.257483 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_085f78183e1e4000a83ccbb0ed350340 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.263981 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 78a6470720984761a8606abeafbef992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.271564 885 tablet_service.cc:1511] Processing CreateTablet for tablet 06904a54b1c5463da31af0bc00fb606f (DEFAULT_TABLE table=default.Table_577ff393d2af447a915f4460ab7692f1 [id=96336409bde3400fb18cb06e0d84e99c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.272318 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06904a54b1c5463da31af0bc00fb606f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.272876 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 93d878ae0ddb42319560f74e0a490321 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:56.273262 887 tablet_service.cc:1511] Processing CreateTablet for tablet 141665fcd59048bf926acd0287f1853e (DEFAULT_TABLE table=default.Table_0ec687e90aea4825b28b36eda22678fe [id=c551b1aa1f2e4eaab4ce7c6727c05465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.274073 891 tablet_service.cc:1511] Processing CreateTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 (DEFAULT_TABLE table=default.Table_1b644b185b194940a8a8a90595ae9e90 [id=8e18fbfbd36340fabddf08dc99fea437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.274487 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 141665fcd59048bf926acd0287f1853e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.274917 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff7ed926a3ff4e7993864de734ca3c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.276417 893 tablet_service.cc:1511] Processing CreateTablet for tablet 5683d7366de84653ace56f0edcd3818c (DEFAULT_TABLE table=default.Table_eb5ee86732cf4963809c3838e7654571 [id=93dfee1a134944188b041e9281fa16c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.277974 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5683d7366de84653ace56f0edcd3818c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:56.279246 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 83e62b7af9b847b7a9005e4345905c0d consensus state has no leader
I20260505 07:53:56.279215 1018 tablet_service.cc:1511] Processing CreateTablet for tablet 83e62b7af9b847b7a9005e4345905c0d (DEFAULT_TABLE table=default.Table_0bc65d0acfcb4ab2a829de8f1d831f9c [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.279564 1181 raft_consensus.cc:359] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.279760 1181 raft_consensus.cc:385] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.279855 1181 raft_consensus.cc:740] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:56.280084 1181 consensus_queue.cc:260] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.280437 1016 tablet_service.cc:1558] Processing DeleteTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.280596 882 tablet_service.cc:1511] Processing CreateTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c (DEFAULT_TABLE table=default.Table_01a5eabcd2ff41fca0d5c94e26775754 [id=0a65eba2d4074216964f45dbb0217d7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.280840 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet 78a6470720984761a8606abeafbef992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.280941 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0459f2d45a2d44f1a73e0a0fff53866c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.281756 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet ccdde3e5a13e436d9f2328831e155adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.283906 886 tablet_service.cc:1511] Processing CreateTablet for tablet 0baf83b155494fddb3c25144608706f7 (DEFAULT_TABLE table=default.Table_5edc57ff557343629eb69ed0fe818ded [id=a92ad827e74542d98f9825d5aa198670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.284230 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0baf83b155494fddb3c25144608706f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.284248 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_3c9a0977f3244728b0e4de172b1559d1" } new_table_name: "default.Table_ad09d6b42bc344c6ba96f4322d75601f" modify_external_catalogs: true
I20260505 07:53:56.292230 890 tablet_service.cc:1511] Processing CreateTablet for tablet 02412333633a4cdebe95a144fa6c363c (DEFAULT_TABLE table=default.Table_0654ae2daf60435697eeca15ba0c19d7 [id=13d54930c52f4e9eae01bc80a3f1d565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.292496 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 003cd4472b424aacbba8c5f89900e228
I20260505 07:53:56.292533 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02412333633a4cdebe95a144fa6c363c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.293581 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 1852f7f4ff594639908b03730b68fe86
I20260505 07:53:56.294036 592 catalog_manager.cc:6741] ReplaceTablet: tablet 1852f7f4ff594639908b03730b68fe86 deleted and replaced by tablet aab3850799254dda9716fd5bf9404159
I20260505 07:53:56.294261 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 1ef4f516e64a44fd8cd33674fb808067
I20260505 07:53:56.294719 595 catalog_manager.cc:6741] ReplaceTablet: tablet 1ef4f516e64a44fd8cd33674fb808067 deleted and replaced by tablet 42360f8ee4ff47c2af8d287f792207de
I20260505 07:53:56.294878 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 1838f89dc14a47f99f95f4cc5fdefd70
I20260505 07:53:56.295288 599 catalog_manager.cc:6741] ReplaceTablet: tablet 1838f89dc14a47f99f95f4cc5fdefd70 deleted and replaced by tablet 4af9def97a454f3f9b63d637db42c110
I20260505 07:53:56.296376 1229 raft_consensus.cc:493] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:56.296535 883 tablet_service.cc:1558] Processing DeleteTablet for tablet 1852f7f4ff594639908b03730b68fe86 with delete_type TABLET_DATA_DELETED (replaced by tablet aab3850799254dda9716fd5bf9404159) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.296759 884 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 with delete_type TABLET_DATA_DELETED (replaced by tablet 42360f8ee4ff47c2af8d287f792207de) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.296957 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 with delete_type TABLET_DATA_DELETED (replaced by tablet 4af9def97a454f3f9b63d637db42c110) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.297266 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 003cd4472b424aacbba8c5f89900e228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.297766 1229 raft_consensus.cc:515] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:56.298424 1229 leader_election.cc:290] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:56.299654 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a79578757824bfc9df129e3475e7af3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:56.299701 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7219f9896fd94187ae714635077110af consensus state has no leader
I20260505 07:53:56.300325 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 06904a54b1c5463da31af0bc00fb606f (DEFAULT_TABLE table=default.Table_577ff393d2af447a915f4460ab7692f1 [id=96336409bde3400fb18cb06e0d84e99c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.300812 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f57568e1c4c4573ac18d9f621eeb4c7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:56.301271 856 leader_election.cc:343] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:56.301918 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_577ff393d2af447a915f4460ab7692f1" } modify_external_catalogs: true
I20260505 07:53:56.302074 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_577ff393d2af447a915f4460ab7692f1" } modify_external_catalogs: true
I20260505 07:53:56.302649 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ac5741221f7e4f54bffba1c88d5cf5c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.303144 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.304383 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c61fca437ddb413e934595a23b52f40f" } modify_external_catalogs: true
I20260505 07:53:56.304558 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c61fca437ddb413e934595a23b52f40f" } modify_external_catalogs: true
I20260505 07:53:56.305019 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 93d878ae0ddb42319560f74e0a490321
I20260505 07:53:56.309499 594 catalog_manager.cc:6741] ReplaceTablet: tablet 93d878ae0ddb42319560f74e0a490321 deleted and replaced by tablet bc9ae6404dd64bd8bf4898768e94e4f1
I20260505 07:53:56.310335 886 tablet_service.cc:1558] Processing DeleteTablet for tablet 93d878ae0ddb42319560f74e0a490321 with delete_type TABLET_DATA_DELETED (replaced by tablet bc9ae6404dd64bd8bf4898768e94e4f1) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.315052 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 5683d7366de84653ace56f0edcd3818c (DEFAULT_TABLE table=default.Table_eb5ee86732cf4963809c3838e7654571 [id=93dfee1a134944188b041e9281fa16c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.317001 1021 tablet_service.cc:1511] Processing CreateTablet for tablet c822fafe72544ca99fde910c8e764755 (DEFAULT_TABLE table=default.Table_3a027f114e1e4d5fabe7ddc8f584e3ec [id=85e91b69517e415194bc6c0dd62ccf9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.318492 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9494c3b6a99487596a030856b371b68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.318517 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5f57568e1c4c4573ac18d9f621eeb4c7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:56.318836 1172 tablet_bootstrap.cc:492] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:56.318864 755 tablet_service.cc:1511] Processing CreateTablet for tablet 06904a54b1c5463da31af0bc00fb606f (DEFAULT_TABLE table=default.Table_577ff393d2af447a915f4460ab7692f1 [id=96336409bde3400fb18cb06e0d84e99c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.318984 1172 ts_tablet_manager.cc:1403] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.114s user 0.005s sys 0.012s
I20260505 07:53:56.319073 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a79578757824bfc9df129e3475e7af3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:56.319185 854 leader_election.cc:343] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:56.319334 854 leader_election.cc:304] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:56.320086 1172 raft_consensus.cc:359] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.320298 1172 raft_consensus.cc:385] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.320376 1172 raft_consensus.cc:740] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:56.320617 1172 consensus_queue.cc:260] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.321702 1229 raft_consensus.cc:2749] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:56.325090 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 9862b87d74974dc49d7dc9049b2a694a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.328449 891 tablet_service.cc:1511] Processing CreateTablet for tablet 8b57c1c618484d5d86a846fb982a2240 (DEFAULT_TABLE table=default.Table_085f78183e1e4000a83ccbb0ed350340 [id=037074511cd8426487cfdb5298d1ee8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.329038 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b57c1c618484d5d86a846fb982a2240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.329687 893 tablet_service.cc:1511] Processing CreateTablet for tablet 62d4bc3154704ba2bfde83bca10f812d (DEFAULT_TABLE table=default.Table_82a545edb422426ab7eaeb4ae68a1a3f [id=5a1cf8871df74960bb29f3288ea4826c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.330022 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d4bc3154704ba2bfde83bca10f812d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.334925 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 7585f00119e14006990ed551667b4486
I20260505 07:53:56.336612 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 06904a54b1c5463da31af0bc00fb606f
I20260505 07:53:56.338444 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 06904a54b1c5463da31af0bc00fb606f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.338693 886 tablet_service.cc:1511] Processing CreateTablet for tablet 55d11a9975854940a0be1693508f0841 (DEFAULT_TABLE table=default.Table_659ddda68e194bb7ae10409bbef074c6 [id=f6e2d9ac0e374944988e8ef7f88fa4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.338739 897 tablet_service.cc:1511] Processing CreateTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf (DEFAULT_TABLE table=default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5 [id=c10b484439c443368141ceb4cf43365f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.339082 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55d11a9975854940a0be1693508f0841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.340794 1176 ts_tablet_manager.cc:1434] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.086s user 0.001s sys 0.001s
I20260505 07:53:56.341352 896 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c (DEFAULT_TABLE table=default.Table_27c52b9b060d4dd5a80799a987d5b7c9 [id=c41c8ca8836b4b4d96bc2f1129279f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.341681 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba2cc218300472cb5fd6dc8cfb7811c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.343853 899 tablet_service.cc:1511] Processing CreateTablet for tablet ba2494780bdd4b6dae6fbdf71125da88 (DEFAULT_TABLE table=default.Table_063cc9f08ab54458b9e277ca65091b11 [id=4f094780dd764a01bb6ba8c7b08d6ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.344199 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2494780bdd4b6dae6fbdf71125da88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.345386 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b124eda66364b7a81426934b98bca48 with delete_type TABLET_DATA_DELETED (replaced by tablet 3781d0bbcd1b4189bd836eeae4236f40) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.345695 883 tablet_service.cc:1511] Processing CreateTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee (DEFAULT_TABLE table=default.Table_92db3a6ebf824942966e677a2cd23ce1 [id=4890bd76a6f04392848038377244273d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.346045 884 tablet_service.cc:1558] Processing DeleteTablet for tablet 7585f00119e14006990ed551667b4486 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.346279 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bde59d52fd234343b1ca13a7d4aae6bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.351294 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8c9eaaef250461ca7a37d0cf42b07ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.353801 748 tablet_service.cc:1511] Processing CreateTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 (DEFAULT_TABLE table=default.Table_1b644b185b194940a8a8a90595ae9e90 [id=8e18fbfbd36340fabddf08dc99fea437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.354139 754 tablet_service.cc:1511] Processing CreateTablet for tablet 5683d7366de84653ace56f0edcd3818c (DEFAULT_TABLE table=default.Table_eb5ee86732cf4963809c3838e7654571 [id=93dfee1a134944188b041e9281fa16c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.356078 1176 tablet_bootstrap.cc:492] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:56.358132 901 tablet_service.cc:1511] Processing CreateTablet for tablet 4af9def97a454f3f9b63d637db42c110 (DEFAULT_TABLE table=default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.358255 898 tablet_service.cc:1511] Processing CreateTablet for tablet aab3850799254dda9716fd5bf9404159 (DEFAULT_TABLE table=default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.358469 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af9def97a454f3f9b63d637db42c110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.358721 884 tablet_service.cc:1511] Processing CreateTablet for tablet 42360f8ee4ff47c2af8d287f792207de (DEFAULT_TABLE table=default.Table_8cdf9d3d3e6349f4b8d3536f78451100 [id=2c91868ddba84103b9a5c78201842a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.359030 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42360f8ee4ff47c2af8d287f792207de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.359727 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab3850799254dda9716fd5bf9404159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.361302 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06904a54b1c5463da31af0bc00fb606f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.361552 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff7ed926a3ff4e7993864de734ca3c50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.361526 745 tablet_service.cc:1511] Processing CreateTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c (DEFAULT_TABLE table=default.Table_01a5eabcd2ff41fca0d5c94e26775754 [id=0a65eba2d4074216964f45dbb0217d7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.361868 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0459f2d45a2d44f1a73e0a0fff53866c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.363816 895 tablet_service.cc:1511] Processing CreateTablet for tablet 12501fdf2cc5491cb1a06db490e70810 (DEFAULT_TABLE table=default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.361040 760 tablet_service.cc:1511] Processing CreateTablet for tablet 141665fcd59048bf926acd0287f1853e (DEFAULT_TABLE table=default.Table_0ec687e90aea4825b28b36eda22678fe [id=c551b1aa1f2e4eaab4ce7c6727c05465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.364182 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12501fdf2cc5491cb1a06db490e70810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.370662 744 tablet_service.cc:1511] Processing CreateTablet for tablet 0baf83b155494fddb3c25144608706f7 (DEFAULT_TABLE table=default.Table_5edc57ff557343629eb69ed0fe818ded [id=a92ad827e74542d98f9825d5aa198670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.371116 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 with delete_type TABLET_DATA_DELETED (replaced by tablet 4af9def97a454f3f9b63d637db42c110) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.371865 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5683d7366de84653ace56f0edcd3818c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.372663 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 93d878ae0ddb42319560f74e0a490321 with delete_type TABLET_DATA_DELETED (replaced by tablet bc9ae6404dd64bd8bf4898768e94e4f1) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.373638 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 1852f7f4ff594639908b03730b68fe86 with delete_type TABLET_DATA_DELETED (replaced by tablet aab3850799254dda9716fd5bf9404159) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.374647 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 003cd4472b424aacbba8c5f89900e228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.376026 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 with delete_type TABLET_DATA_DELETED (replaced by tablet 42360f8ee4ff47c2af8d287f792207de) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.375982 887 tablet_service.cc:1511] Processing CreateTablet for tablet c1a61c1d03d544c8aa01537ead97f83c (DEFAULT_TABLE table=default.Table_f6734bb37ebf474982961491f124f0f1 [id=cdb574c7a4e14a5ea7f3c8b036a8c336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.376163 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0baf83b155494fddb3c25144608706f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.376466 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a61c1d03d544c8aa01537ead97f83c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.378238 1176 tablet_bootstrap.cc:654] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.381780 1171 ts_tablet_manager.cc:1929] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:56.381901 1171 log.cc:1199] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/25a74b65d5d6459293ab4461f4d34cf2
I20260505 07:53:56.382244 1171 ts_tablet_manager.cc:1950] T 25a74b65d5d6459293ab4461f4d34cf2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:56.383926 882 tablet_service.cc:1511] Processing CreateTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 (DEFAULT_TABLE table=default.Table_f8f7a899e8c94e75958bd11fed643bc6 [id=a3cea9a70f4147b7bf5b630980c94c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.384298 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0f2f46d0e7840458cb992ce2a2dbea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.385681 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 141665fcd59048bf926acd0287f1853e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.384616 743 tablet_service.cc:1511] Processing CreateTablet for tablet 02412333633a4cdebe95a144fa6c363c (DEFAULT_TABLE table=default.Table_0654ae2daf60435697eeca15ba0c19d7 [id=13d54930c52f4e9eae01bc80a3f1d565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.386986 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 003cd4472b424aacbba8c5f89900e228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.387218 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83e62b7af9b847b7a9005e4345905c0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.387174 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c (DEFAULT_TABLE table=default.Table_01a5eabcd2ff41fca0d5c94e26775754 [id=0a65eba2d4074216964f45dbb0217d7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.387553 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0459f2d45a2d44f1a73e0a0fff53866c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.387567 890 tablet_service.cc:1511] Processing CreateTablet for tablet 46120e7a5dcb482f8f93fc2aa5219aff (DEFAULT_TABLE table=default.Table_59819a75045240de818a627f6036f8e3 [id=c019a4cae5d0458b85da8106e6819924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.388007 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46120e7a5dcb482f8f93fc2aa5219aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.388048 752 tablet_service.cc:1511] Processing CreateTablet for tablet 55d11a9975854940a0be1693508f0841 (DEFAULT_TABLE table=default.Table_659ddda68e194bb7ae10409bbef074c6 [id=f6e2d9ac0e374944988e8ef7f88fa4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.388033 1029 tablet_service.cc:1511] Processing CreateTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 (DEFAULT_TABLE table=default.Table_1b644b185b194940a8a8a90595ae9e90 [id=8e18fbfbd36340fabddf08dc99fea437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.388927 762 tablet_service.cc:1511] Processing CreateTablet for tablet c1a61c1d03d544c8aa01537ead97f83c (DEFAULT_TABLE table=default.Table_f6734bb37ebf474982961491f124f0f1 [id=cdb574c7a4e14a5ea7f3c8b036a8c336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.389335 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 141665fcd59048bf926acd0287f1853e (DEFAULT_TABLE table=default.Table_0ec687e90aea4825b28b36eda22678fe [id=c551b1aa1f2e4eaab4ce7c6727c05465]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.389456 759 tablet_service.cc:1511] Processing CreateTablet for tablet 8b57c1c618484d5d86a846fb982a2240 (DEFAULT_TABLE table=default.Table_085f78183e1e4000a83ccbb0ed350340 [id=037074511cd8426487cfdb5298d1ee8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.389958 757 tablet_service.cc:1511] Processing CreateTablet for tablet 62d4bc3154704ba2bfde83bca10f812d (DEFAULT_TABLE table=default.Table_82a545edb422426ab7eaeb4ae68a1a3f [id=5a1cf8871df74960bb29f3288ea4826c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.392432 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02412333633a4cdebe95a144fa6c363c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:56.394510 717 consensus_peers.cc:597] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:53:56.399214 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55d11a9975854940a0be1693508f0841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.401085 899 tablet_service.cc:1511] Processing CreateTablet for tablet bc9ae6404dd64bd8bf4898768e94e4f1 (DEFAULT_TABLE table=default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.401796 921 raft_consensus.cc:1275] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Refusing update from remote peer eac7e1e8caba43cdb4df80cbb80e891c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260505 07:53:56.404400 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9ae6404dd64bd8bf4898768e94e4f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.405169 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_43b383b7a58b493c8f5b2d9f3038a2e2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.405838 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_43b383b7a58b493c8f5b2d9f3038a2e2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.406669 1181 ts_tablet_manager.cc:1434] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.166s user 0.000s sys 0.011s
I20260505 07:53:56.407388 1181 tablet_bootstrap.cc:492] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:56.409221 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 25a74b65d5d6459293ab4461f4d34cf2 (table default.Table_eb39510acd7a40b6a7f20bd09e300ff7 [id=94a6d460807e43768c78bc319239b699]) successfully deleted
I20260505 07:53:56.411469 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d4bc3154704ba2bfde83bca10f812d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:56.413331 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 4a79578757824bfc9df129e3475e7af3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a79578757824bfc9df129e3475e7af3 already in progress: creating tablet
W20260505 07:53:56.414522 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 1852f7f4ff594639908b03730b68fe86 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1852f7f4ff594639908b03730b68fe86 already in progress: creating tablet
I20260505 07:53:56.415522 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff7ed926a3ff4e7993864de734ca3c50. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:56.415684 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 1ef4f516e64a44fd8cd33674fb808067 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ef4f516e64a44fd8cd33674fb808067 already in progress: creating tablet
I20260505 07:53:56.416160 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b57c1c618484d5d86a846fb982a2240. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:56.417152 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 1838f89dc14a47f99f95f4cc5fdefd70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1838f89dc14a47f99f95f4cc5fdefd70 already in progress: creating tablet
I20260505 07:53:56.417830 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5683d7366de84653ace56f0edcd3818c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.418923 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c822fafe72544ca99fde910c8e764755. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:56.419096 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 003cd4472b424aacbba8c5f89900e228 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 003cd4472b424aacbba8c5f89900e228 already in progress: creating tablet
I20260505 07:53:56.419605 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 067c32ecccb44bca8bf9cc3f6c229326. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.421226 1273 consensus_queue.cc:1048] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
W20260505 07:53:56.422078 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 93d878ae0ddb42319560f74e0a490321 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 93d878ae0ddb42319560f74e0a490321 already in progress: creating tablet
W20260505 07:53:56.422775 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 9862b87d74974dc49d7dc9049b2a694a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9862b87d74974dc49d7dc9049b2a694a already in progress: creating tablet
I20260505 07:53:56.423028 1172 ts_tablet_manager.cc:1434] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.104s user 0.005s sys 0.001s
W20260505 07:53:56.424636 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 06904a54b1c5463da31af0bc00fb606f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06904a54b1c5463da31af0bc00fb606f already in progress: creating tablet
I20260505 07:53:56.424670 1172 tablet_bootstrap.cc:492] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:53:56.425218 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 7585f00119e14006990ed551667b4486 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7585f00119e14006990ed551667b4486 already in progress: creating tablet
I20260505 07:53:56.426647 1172 tablet_bootstrap.cc:654] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.426995 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06904a54b1c5463da31af0bc00fb606f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.429117 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_5edc57ff557343629eb69ed0fe818ded" } modify_external_catalogs: true
I20260505 07:53:56.429314 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_5edc57ff557343629eb69ed0fe818ded" } modify_external_catalogs: true
I20260505 07:53:56.429562 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_31dd1382f8b4494e9e13ac5b3ac18aa0" } new_table_name: "default.Table_44c3bcc031ad40a796ea43c44863d46f" modify_external_catalogs: true
I20260505 07:53:56.433203 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_dcf56067c2654eff8773e49d0ddf5427"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:56.433476 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_477749a4e38f41c48eb5d2fd5a7247e7" } new_table_name: "default.Table_087f86f68d1a411a8aba986365a3b2fe" modify_external_catalogs: true
W20260505 07:53:56.433774 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dcf56067c2654eff8773e49d0ddf5427 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.430845 747 tablet_service.cc:1511] Processing CreateTablet for tablet 46120e7a5dcb482f8f93fc2aa5219aff (DEFAULT_TABLE table=default.Table_59819a75045240de818a627f6036f8e3 [id=c019a4cae5d0458b85da8106e6819924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.437170 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 141665fcd59048bf926acd0287f1853e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.439100 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a61c1d03d544c8aa01537ead97f83c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.439684 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 0baf83b155494fddb3c25144608706f7 (DEFAULT_TABLE table=default.Table_5edc57ff557343629eb69ed0fe818ded [id=a92ad827e74542d98f9825d5aa198670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.443136 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0baf83b155494fddb3c25144608706f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.441102 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 9862b87d74974dc49d7dc9049b2a694a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:56.448807 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 571257bbb7e94315b60d84d6aa4917b1 consensus state has no leader
I20260505 07:53:56.452481 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 0baf83b155494fddb3c25144608706f7
I20260505 07:53:56.453925 885 tablet_service.cc:1511] Processing CreateTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 (DEFAULT_TABLE table=default.Table_43b383b7a58b493c8f5b2d9f3038a2e2 [id=5154509860404937ba84ef134bb50cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.454154 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 0baf83b155494fddb3c25144608706f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.454659 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.456564 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_113632d390cc461ca5cc897ef5f6568a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.457063 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_113632d390cc461ca5cc897ef5f6568a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.457734 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6c975d29ebde41ecb90fdaf8c72b2997"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.458197 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c975d29ebde41ecb90fdaf8c72b2997 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.462018 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 02412333633a4cdebe95a144fa6c363c (DEFAULT_TABLE table=default.Table_0654ae2daf60435697eeca15ba0c19d7 [id=13d54930c52f4e9eae01bc80a3f1d565]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.462404 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02412333633a4cdebe95a144fa6c363c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.463548 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 1852f7f4ff594639908b03730b68fe86 with delete_type TABLET_DATA_DELETED (replaced by tablet aab3850799254dda9716fd5bf9404159) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.463544 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 with delete_type TABLET_DATA_DELETED (replaced by tablet 4af9def97a454f3f9b63d637db42c110) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.463877 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 93d878ae0ddb42319560f74e0a490321 with delete_type TABLET_DATA_DELETED (replaced by tablet bc9ae6404dd64bd8bf4898768e94e4f1) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.464210 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 8b57c1c618484d5d86a846fb982a2240 (DEFAULT_TABLE table=default.Table_085f78183e1e4000a83ccbb0ed350340 [id=037074511cd8426487cfdb5298d1ee8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.464586 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b57c1c618484d5d86a846fb982a2240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.465364 1019 tablet_service.cc:1511] Processing CreateTablet for tablet c1a61c1d03d544c8aa01537ead97f83c (DEFAULT_TABLE table=default.Table_f6734bb37ebf474982961491f124f0f1 [id=cdb574c7a4e14a5ea7f3c8b036a8c336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.465710 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1a61c1d03d544c8aa01537ead97f83c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.466430 1016 tablet_service.cc:1511] Processing CreateTablet for tablet 46120e7a5dcb482f8f93fc2aa5219aff (DEFAULT_TABLE table=default.Table_59819a75045240de818a627f6036f8e3 [id=c019a4cae5d0458b85da8106e6819924]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.466565 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46120e7a5dcb482f8f93fc2aa5219aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.466624 1033 tablet_service.cc:1558] Processing DeleteTablet for tablet 003cd4472b424aacbba8c5f89900e228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.467269 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46120e7a5dcb482f8f93fc2aa5219aff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.467378 1033 tablet_service.cc:1511] Processing CreateTablet for tablet ba2494780bdd4b6dae6fbdf71125da88 (DEFAULT_TABLE table=default.Table_063cc9f08ab54458b9e277ca65091b11 [id=4f094780dd764a01bb6ba8c7b08d6ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.467998 1025 tablet_service.cc:1511] Processing CreateTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 (DEFAULT_TABLE table=default.Table_f8f7a899e8c94e75958bd11fed643bc6 [id=a3cea9a70f4147b7bf5b630980c94c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.468333 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0f2f46d0e7840458cb992ce2a2dbea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.468410 1181 tablet_bootstrap.cc:654] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.460244 1368 raft_consensus.cc:493] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:56.468832 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c (DEFAULT_TABLE table=default.Table_27c52b9b060d4dd5a80799a987d5b7c9 [id=c41c8ca8836b4b4d96bc2f1129279f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.469137 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba2cc218300472cb5fd6dc8cfb7811c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.472044 1171 tablet_replica.cc:333] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:56.472229 1171 raft_consensus.cc:2243] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:56.472406 1171 raft_consensus.cc:2272] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:56.473137 1171 ts_tablet_manager.cc:1916] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:56.476032 1176 tablet_bootstrap.cc:492] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:56.482869 1022 tablet_service.cc:1511] Processing CreateTablet for tablet 55d11a9975854940a0be1693508f0841 (DEFAULT_TABLE table=default.Table_659ddda68e194bb7ae10409bbef074c6 [id=f6e2d9ac0e374944988e8ef7f88fa4be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.483189 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55d11a9975854940a0be1693508f0841. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.484300 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 with delete_type TABLET_DATA_DELETED (replaced by tablet 42360f8ee4ff47c2af8d287f792207de) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.485201 1176 ts_tablet_manager.cc:1403] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.120s user 0.004s sys 0.001s
I20260505 07:53:56.485848 754 tablet_service.cc:1511] Processing CreateTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 (DEFAULT_TABLE table=default.Table_f8f7a899e8c94e75958bd11fed643bc6 [id=a3cea9a70f4147b7bf5b630980c94c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.486022 1029 tablet_service.cc:1511] Processing CreateTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf (DEFAULT_TABLE table=default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5 [id=c10b484439c443368141ceb4cf43365f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.486227 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0f2f46d0e7840458cb992ce2a2dbea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.486251 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1da2218f525749af888a9da48524bc11" } new_table_name: "default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c" modify_external_catalogs: true
I20260505 07:53:56.486375 599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f0316c88590245aea8815b3b08e89f45" } modify_external_catalogs: true
I20260505 07:53:56.486557 599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f0316c88590245aea8815b3b08e89f45" } modify_external_catalogs: true
I20260505 07:53:56.487215 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_b3b9364eea524724b176de87f62eb49e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.487712 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b3b9364eea524724b176de87f62eb49e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.489804 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_cb21f88a808c477b8a55a83e7507b951"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.490317 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cb21f88a808c477b8a55a83e7507b951 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.492082 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_3e9abeb1ce814ef2840df1ce519f1768"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:56.492257 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_09561894665e45a98f269fd1d764b78a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.492555 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3e9abeb1ce814ef2840df1ce519f1768 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:56.492754 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09561894665e45a98f269fd1d764b78a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.492091 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_9eb304c401574e9998ad4e8cb54f258b" } new_table_name: "default.Table_f582c0d06a754ba7ad756b60cfde5174" modify_external_catalogs: true
I20260505 07:53:56.495420 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_94e6f1ee4ab947d8bc9864ab0c8bd524"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.496022 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94e6f1ee4ab947d8bc9864ab0c8bd524 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.496615 1178 ts_tablet_manager.cc:1929] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:56.496731 1178 log.cc:1199] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/31b151ff45804fbaa76cfbec86c6fd7a
I20260505 07:53:56.497089 1178 ts_tablet_manager.cc:1950] T 31b151ff45804fbaa76cfbec86c6fd7a P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:56.501250 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 06904a54b1c5463da31af0bc00fb606f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.501377 749 tablet_service.cc:1511] Processing CreateTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c (DEFAULT_TABLE table=default.Table_27c52b9b060d4dd5a80799a987d5b7c9 [id=c41c8ca8836b4b4d96bc2f1129279f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.501435 899 tablet_service.cc:1511] Processing CreateTablet for tablet 3f1feb0387bc4afead72346398dbe9e5 (DEFAULT_TABLE table=default.Table_dcf56067c2654eff8773e49d0ddf5427 [id=a9ef91bfcff640abb23140249a59fcf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.501605 1023 tablet_service.cc:1511] Processing CreateTablet for tablet bc9ae6404dd64bd8bf4898768e94e4f1 (DEFAULT_TABLE table=default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.501814 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f1feb0387bc4afead72346398dbe9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.501870 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ba2cc218300472cb5fd6dc8cfb7811c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.508991 1020 tablet_service.cc:1511] Processing CreateTablet for tablet 62d4bc3154704ba2bfde83bca10f812d (DEFAULT_TABLE table=default.Table_82a545edb422426ab7eaeb4ae68a1a3f [id=5a1cf8871df74960bb29f3288ea4826c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.509604 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2494780bdd4b6dae6fbdf71125da88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.510133 746 tablet_service.cc:1511] Processing CreateTablet for tablet ba2494780bdd4b6dae6fbdf71125da88 (DEFAULT_TABLE table=default.Table_063cc9f08ab54458b9e277ca65091b11 [id=4f094780dd764a01bb6ba8c7b08d6ef0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.510598 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba2494780bdd4b6dae6fbdf71125da88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.512303 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62d4bc3154704ba2bfde83bca10f812d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.513039 1030 tablet_service.cc:1511] Processing CreateTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee (DEFAULT_TABLE table=default.Table_92db3a6ebf824942966e677a2cd23ce1 [id=4890bd76a6f04392848038377244273d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.514457 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8c9eaaef250461ca7a37d0cf42b07ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.517472 600 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_92db3a6ebf824942966e677a2cd23ce1" } modify_external_catalogs: true
I20260505 07:53:56.517693 600 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_92db3a6ebf824942966e677a2cd23ce1" } modify_external_catalogs: true
W20260505 07:53:56.521884 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6ca2852cee84476dbe5e5072a43087a1 consensus state has no leader
I20260505 07:53:56.526942 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 7585f00119e14006990ed551667b4486 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.527312 750 tablet_service.cc:1511] Processing CreateTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf (DEFAULT_TABLE table=default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5 [id=c10b484439c443368141ceb4cf43365f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.527420 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_cf18ef8e6a56482883dba2744997604a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.527997 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf18ef8e6a56482883dba2744997604a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.529240 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_113632d390cc461ca5cc897ef5f6568a" } new_table_name: "default.Table_a24821d9d3f14fee9ce992364d12721e" modify_external_catalogs: true
I20260505 07:53:56.535491 1175 ts_tablet_manager.cc:1929] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:56.538970 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0e71e56d0528481eb81fbca5fe427f53" } modify_external_catalogs: true
I20260505 07:53:56.539113 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0e71e56d0528481eb81fbca5fe427f53" } modify_external_catalogs: true
I20260505 07:53:56.543880 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bde59d52fd234343b1ca13a7d4aae6bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.546095 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_dcf56067c2654eff8773e49d0ddf5427" } modify_external_catalogs: true
I20260505 07:53:56.546281 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_dcf56067c2654eff8773e49d0ddf5427" } modify_external_catalogs: true
I20260505 07:53:56.547935 1176 raft_consensus.cc:359] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.548130 1176 raft_consensus.cc:385] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.548218 761 tablet_service.cc:1511] Processing CreateTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee (DEFAULT_TABLE table=default.Table_92db3a6ebf824942966e677a2cd23ce1 [id=4890bd76a6f04392848038377244273d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.548578 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8c9eaaef250461ca7a37d0cf42b07ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.548775 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 42360f8ee4ff47c2af8d287f792207de (DEFAULT_TABLE table=default.Table_8cdf9d3d3e6349f4b8d3536f78451100 [id=2c91868ddba84103b9a5c78201842a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.550066 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 7585f00119e14006990ed551667b4486 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.550467 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 12501fdf2cc5491cb1a06db490e70810 (DEFAULT_TABLE table=default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.555655 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bde59d52fd234343b1ca13a7d4aae6bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.558197 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 4af9def97a454f3f9b63d637db42c110 (DEFAULT_TABLE table=default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.564600 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet b7cce14283e7405690a15e26be8a45b3 with delete_type TABLET_DATA_DELETED (replaced by tablet 6429919aba17447dbdb393176360bbb0) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.565064 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 (DEFAULT_TABLE table=default.Table_43b383b7a58b493c8f5b2d9f3038a2e2 [id=5154509860404937ba84ef134bb50cef]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:56.566183 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bd581a6799a14a8dafb80d411e8936f3 consensus state has no leader
I20260505 07:53:56.567837 751 tablet_service.cc:1511] Processing CreateTablet for tablet 12501fdf2cc5491cb1a06db490e70810 (DEFAULT_TABLE table=default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.568738 756 tablet_service.cc:1511] Processing CreateTablet for tablet 42360f8ee4ff47c2af8d287f792207de (DEFAULT_TABLE table=default.Table_8cdf9d3d3e6349f4b8d3536f78451100 [id=2c91868ddba84103b9a5c78201842a9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.569046 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42360f8ee4ff47c2af8d287f792207de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.569810 1024 tablet_service.cc:1511] Processing CreateTablet for tablet aab3850799254dda9716fd5bf9404159 (DEFAULT_TABLE table=default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.570183 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab3850799254dda9716fd5bf9404159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.571161 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9ae6404dd64bd8bf4898768e94e4f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.572021 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af9def97a454f3f9b63d637db42c110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.542974 1368 raft_consensus.cc:515] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.572947 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12501fdf2cc5491cb1a06db490e70810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.573734 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de9e0fe8eb7477ab1cfa64b808575d8" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:56.575066 717 leader_election.cc:343] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:56.577474 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de9e0fe8eb7477ab1cfa64b808575d8" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:53:56.577780 920 raft_consensus.cc:2468] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:53:56.579514 1176 raft_consensus.cc:740] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:56.579861 1176 consensus_queue.cc:260] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.581347 1018 tablet_service.cc:1558] Processing DeleteTablet for tablet 0baf83b155494fddb3c25144608706f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.582685 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 7e707143a4e64725aa66db05b24999a6
I20260505 07:53:56.583866 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e707143a4e64725aa66db05b24999a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.585800 748 tablet_service.cc:1558] Processing DeleteTablet for tablet 06904a54b1c5463da31af0bc00fb606f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.570056 758 tablet_service.cc:1511] Processing CreateTablet for tablet bc9ae6404dd64bd8bf4898768e94e4f1 (DEFAULT_TABLE table=default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.587760 1175 log.cc:1199] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/8713b273c7b8438d8749f595ee5f1629
I20260505 07:53:56.590808 1176 ts_tablet_manager.cc:1434] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.105s user 0.002s sys 0.000s
I20260505 07:53:56.593226 1175 ts_tablet_manager.cc:1950] T 8713b273c7b8438d8749f595ee5f1629 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:56.599324 600 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet d8c9eaaef250461ca7a37d0cf42b07ee
I20260505 07:53:56.599696 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.601126 1366 raft_consensus.cc:493] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:56.601255 1366 raft_consensus.cc:515] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.602063 1366 leader_election.cc:290] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:56.603019 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a9420a81844d2092381123d9297661" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:56.603747 856 leader_election.cc:343] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:56.604214 899 tablet_service.cc:1558] Processing DeleteTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.609622 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a9420a81844d2092381123d9297661" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:56.610213 854 leader_election.cc:343] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:56.610337 854 leader_election.cc:304] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:56.610949 1366 raft_consensus.cc:2749] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:56.611137 744 tablet_service.cc:1558] Processing DeleteTablet for tablet 0baf83b155494fddb3c25144608706f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.612270 744 tablet_service.cc:1511] Processing CreateTablet for tablet 3f1feb0387bc4afead72346398dbe9e5 (DEFAULT_TABLE table=default.Table_dcf56067c2654eff8773e49d0ddf5427 [id=a9ef91bfcff640abb23140249a59fcf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.612457 595 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:53:56.615787 1176 tablet_bootstrap.cc:492] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:53:56.619345 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 08c6ed755fc64080a2310fe9a4e0a68b has no consensus state
I20260505 07:53:56.626984 1175 tablet_replica.cc:333] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:56.627231 1175 raft_consensus.cc:2243] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:56.627446 1175 raft_consensus.cc:2272] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:56.629617 1178 tablet_replica.cc:333] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:56.632254 899 tablet_service.cc:1511] Processing CreateTablet for tablet 08c6ed755fc64080a2310fe9a4e0a68b (DEFAULT_TABLE table=default.Table_a24821d9d3f14fee9ce992364d12721e [id=0b202734a7744102a98cc55ac980603b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.632328 887 tablet_service.cc:1511] Processing CreateTablet for tablet d37dc6df9da5401384a6fc5de868b41c (DEFAULT_TABLE table=default.Table_6c975d29ebde41ecb90fdaf8c72b2997 [id=0a01d809c30041a0a5b803341584f3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.635185 752 tablet_service.cc:1511] Processing CreateTablet for tablet 4af9def97a454f3f9b63d637db42c110 (DEFAULT_TABLE table=default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.636128 755 tablet_service.cc:1511] Processing CreateTablet for tablet aab3850799254dda9716fd5bf9404159 (DEFAULT_TABLE table=default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.636909 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12501fdf2cc5491cb1a06db490e70810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.637367 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9ae6404dd64bd8bf4898768e94e4f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.638433 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 31b151ff45804fbaa76cfbec86c6fd7a (table default.Table_7f23cbfec9794a2bbc9e9dad2d07011c [id=b25c09d8fb8347af83c69e76a523867e]) successfully deleted
W20260505 07:53:56.638705 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 3d6494590052459485059f5dce46f34f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d6494590052459485059f5dce46f34f already in progress: creating tablet
W20260505 07:53:56.638924 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet c6dcbebddcdc4ad9a788c01134e1101c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c6dcbebddcdc4ad9a788c01134e1101c already in progress: creating tablet
W20260505 07:53:56.639148 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 6429919aba17447dbdb393176360bbb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6429919aba17447dbdb393176360bbb0 already in progress: creating tablet
W20260505 07:53:56.639323 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 32c008bd99224a7595cecc607e5496a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32c008bd99224a7595cecc607e5496a9 already in progress: creating tablet
W20260505 07:53:56.639511 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e85b44337f62494c997c0008fc492aab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e85b44337f62494c997c0008fc492aab already in progress: creating tablet
W20260505 07:53:56.641222 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 976359601ad74b149696dc7f555b550c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 976359601ad74b149696dc7f555b550c already in progress: creating tablet
W20260505 07:53:56.641433 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8a60c015136246a3b02f70254c0bb67f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a60c015136246a3b02f70254c0bb67f already in progress: creating tablet
W20260505 07:53:56.641635 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 6da10815e0dc41a1a07b6786882fffd3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6da10815e0dc41a1a07b6786882fffd3 already in progress: creating tablet
W20260505 07:53:56.645183 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 78a6470720984761a8606abeafbef992 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78a6470720984761a8606abeafbef992 already in progress: creating tablet
W20260505 07:53:56.645439 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet ccdde3e5a13e436d9f2328831e155adb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccdde3e5a13e436d9f2328831e155adb already in progress: creating tablet
W20260505 07:53:56.645733 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 4a79578757824bfc9df129e3475e7af3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a79578757824bfc9df129e3475e7af3 already in progress: creating tablet
W20260505 07:53:56.646121 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 1852f7f4ff594639908b03730b68fe86 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1852f7f4ff594639908b03730b68fe86 already in progress: creating tablet
W20260505 07:53:56.646374 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 1838f89dc14a47f99f95f4cc5fdefd70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1838f89dc14a47f99f95f4cc5fdefd70 already in progress: creating tablet
W20260505 07:53:56.646595 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 93d878ae0ddb42319560f74e0a490321 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 93d878ae0ddb42319560f74e0a490321 already in progress: creating tablet
W20260505 07:53:56.646817 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 003cd4472b424aacbba8c5f89900e228 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 003cd4472b424aacbba8c5f89900e228 already in progress: creating tablet
W20260505 07:53:56.647083 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 1ef4f516e64a44fd8cd33674fb808067 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ef4f516e64a44fd8cd33674fb808067 already in progress: creating tablet
W20260505 07:53:56.647349 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 06904a54b1c5463da31af0bc00fb606f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06904a54b1c5463da31af0bc00fb606f already in progress: creating tablet
W20260505 07:53:56.647626 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 7585f00119e14006990ed551667b4486 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7585f00119e14006990ed551667b4486 already in progress: creating tablet
I20260505 07:53:56.647716 757 tablet_service.cc:1511] Processing CreateTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 (DEFAULT_TABLE table=default.Table_43b383b7a58b493c8f5b2d9f3038a2e2 [id=5154509860404937ba84ef134bb50cef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.648541 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.650859 1368 leader_election.cc:290] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:56.651240 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42360f8ee4ff47c2af8d287f792207de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.651549 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 3f1feb0387bc4afead72346398dbe9e5
I20260505 07:53:56.652552 882 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f1feb0387bc4afead72346398dbe9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.653074 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 17327cee69a24e0cb493ff55f948bf6c
I20260505 07:53:56.653975 882 tablet_service.cc:1558] Processing DeleteTablet for tablet 17327cee69a24e0cb493ff55f948bf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:56.655965 718 leader_election.cc:304] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
W20260505 07:53:56.656468 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 77546.2us
I20260505 07:53:56.658569 1368 raft_consensus.cc:2804] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:56.658771 1368 raft_consensus.cc:493] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:56.658937 1368 raft_consensus.cc:3060] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:56.660436 1175 ts_tablet_manager.cc:1916] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:56.662973 1178 raft_consensus.cc:2243] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:56.663173 1178 raft_consensus.cc:2272] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:56.662943 1368 raft_consensus.cc:515] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.663691 1178 ts_tablet_manager.cc:1916] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:56.664867 1176 tablet_bootstrap.cc:654] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.630291 1181 tablet_bootstrap.cc:492] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:56.640349 1032 tablet_service.cc:1511] Processing CreateTablet for tablet d37dc6df9da5401384a6fc5de868b41c (DEFAULT_TABLE table=default.Table_6c975d29ebde41ecb90fdaf8c72b2997 [id=0a01d809c30041a0a5b803341584f3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.668023 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37dc6df9da5401384a6fc5de868b41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.667994 882 tablet_service.cc:1511] Processing CreateTablet for tablet 5191f9a357de4607aab6c7a8b4bcccdb (DEFAULT_TABLE table=default.Table_94e6f1ee4ab947d8bc9864ab0c8bd524 [id=f7b2051b32e240c1856be3f0802b2f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.668756 890 tablet_service.cc:1511] Processing CreateTablet for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (DEFAULT_TABLE table=default.Table_cb21f88a808c477b8a55a83e7507b951 [id=2c06ff8eed31432e87f08e9b59049652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.670497 891 tablet_service.cc:1511] Processing CreateTablet for tablet 8eb7eaff8837485a8ae3ccb44c60471d (DEFAULT_TABLE table=default.Table_3e9abeb1ce814ef2840df1ce519f1768 [id=266ff796c7a74c23816dd9c422e9d005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.670940 886 tablet_service.cc:1511] Processing CreateTablet for tablet d3d11bd19f074bbab101448b45426b7a (DEFAULT_TABLE table=default.Table_cf18ef8e6a56482883dba2744997604a [id=b131fb4265684cd2b6506a7abeb29871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.672291 883 tablet_service.cc:1511] Processing CreateTablet for tablet 1001dd888ecd4166ba217506739f71e8 (DEFAULT_TABLE table=default.Table_09561894665e45a98f269fd1d764b78a [id=048ca74dfef1481d945932e17b785ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.673593 897 tablet_service.cc:1511] Processing CreateTablet for tablet 4258df74ea0841449846cdf1f6e00b9b (DEFAULT_TABLE table=default.Table_b3b9364eea524724b176de87f62eb49e [id=1fd77c8cfe9d40dab5da490774294796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.640601 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 08c6ed755fc64080a2310fe9a4e0a68b (DEFAULT_TABLE table=default.Table_a24821d9d3f14fee9ce992364d12721e [id=0b202734a7744102a98cc55ac980603b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.641347 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 3f1feb0387bc4afead72346398dbe9e5 (DEFAULT_TABLE table=default.Table_dcf56067c2654eff8773e49d0ddf5427 [id=a9ef91bfcff640abb23140249a59fcf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.674572 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08c6ed755fc64080a2310fe9a4e0a68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.674754 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f1feb0387bc4afead72346398dbe9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.674976 1181 ts_tablet_manager.cc:1403] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.268s user 0.004s sys 0.004s
I20260505 07:53:56.675884 1181 raft_consensus.cc:359] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.679656 1181 raft_consensus.cc:385] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.679831 1181 raft_consensus.cc:740] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:56.680734 1181 consensus_queue.cc:260] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.682772 1020 tablet_service.cc:1511] Processing CreateTablet for tablet 4258df74ea0841449846cdf1f6e00b9b (DEFAULT_TABLE table=default.Table_b3b9364eea524724b176de87f62eb49e [id=1fd77c8cfe9d40dab5da490774294796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.683198 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4258df74ea0841449846cdf1f6e00b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.641531 1033 tablet_service.cc:1558] Processing DeleteTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.641674 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e707143a4e64725aa66db05b24999a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.659855 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 17327cee69a24e0cb493ff55f948bf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.659629 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f1feb0387bc4afead72346398dbe9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:56.689443 1172 tablet_bootstrap.cc:492] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:56.689674 1172 ts_tablet_manager.cc:1403] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.265s user 0.024s sys 0.045s
I20260505 07:53:56.690595 1172 raft_consensus.cc:359] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.691495 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 8eb7eaff8837485a8ae3ccb44c60471d (DEFAULT_TABLE table=default.Table_3e9abeb1ce814ef2840df1ce519f1768 [id=266ff796c7a74c23816dd9c422e9d005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.691069 1171 ts_tablet_manager.cc:1929] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:56.691762 1172 raft_consensus.cc:385] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.691892 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eb7eaff8837485a8ae3ccb44c60471d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.691877 1171 log.cc:1199] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/9862b87d74974dc49d7dc9049b2a694a
I20260505 07:53:56.692061 1172 raft_consensus.cc:740] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:56.692292 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab3850799254dda9716fd5bf9404159. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.692298 1172 consensus_queue.cc:260] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.692773 1172 ts_tablet_manager.cc:1434] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260505 07:53:56.694092 1171 ts_tablet_manager.cc:1950] T 9862b87d74974dc49d7dc9049b2a694a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:56.696218 1029 tablet_service.cc:1511] Processing CreateTablet for tablet d3d11bd19f074bbab101448b45426b7a (DEFAULT_TABLE table=default.Table_cf18ef8e6a56482883dba2744997604a [id=b131fb4265684cd2b6506a7abeb29871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.696635 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3d11bd19f074bbab101448b45426b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.698325 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 8713b273c7b8438d8749f595ee5f1629 (table default.Table_aa53055d0748464697a41feba3c4f036 [id=af8d003d14d24f34bd57e804562d1f33]) successfully deleted
W20260505 07:53:56.698647 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 3d6494590052459485059f5dce46f34f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d6494590052459485059f5dce46f34f already in progress: creating tablet
W20260505 07:53:56.698897 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 8a60c015136246a3b02f70254c0bb67f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8a60c015136246a3b02f70254c0bb67f already in progress: creating tablet
W20260505 07:53:56.699138 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet c6dcbebddcdc4ad9a788c01134e1101c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c6dcbebddcdc4ad9a788c01134e1101c already in progress: creating tablet
W20260505 07:53:56.699338 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e85b44337f62494c997c0008fc492aab because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e85b44337f62494c997c0008fc492aab already in progress: creating tablet
I20260505 07:53:56.700151 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 5191f9a357de4607aab6c7a8b4bcccdb (DEFAULT_TABLE table=default.Table_94e6f1ee4ab947d8bc9864ab0c8bd524 [id=f7b2051b32e240c1856be3f0802b2f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.700790 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5191f9a357de4607aab6c7a8b4bcccdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.704926 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4af9def97a454f3f9b63d637db42c110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.706521 1181 ts_tablet_manager.cc:1434] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.030s user 0.000s sys 0.002s
I20260505 07:53:56.711071 1181 tablet_bootstrap.cc:492] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:56.713472 1181 tablet_bootstrap.cc:654] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.713604 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62bf3a6bc0ac42a99a4181f820e3ca87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.717355 1172 tablet_bootstrap.cc:492] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:56.718403 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08c6ed755fc64080a2310fe9a4e0a68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.719514 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f1feb0387bc4afead72346398dbe9e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.719779 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eb7eaff8837485a8ae3ccb44c60471d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.723707 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 1001dd888ecd4166ba217506739f71e8 (DEFAULT_TABLE table=default.Table_09561894665e45a98f269fd1d764b78a [id=048ca74dfef1481d945932e17b785ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.724159 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37dc6df9da5401384a6fc5de868b41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.724449 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1001dd888ecd4166ba217506739f71e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.727845 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5191f9a357de4607aab6c7a8b4bcccdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.728799 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (DEFAULT_TABLE table=default.Table_cb21f88a808c477b8a55a83e7507b951 [id=2c06ff8eed31432e87f08e9b59049652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.729112 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62bf3a6bc0ac42a99a4181f820e3ca87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.730618 1175 ts_tablet_manager.cc:1929] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:56.730751 1175 log.cc:1199] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/5b124eda66364b7a81426934b98bca48
I20260505 07:53:56.731822 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3d11bd19f074bbab101448b45426b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.732010 1175 ts_tablet_manager.cc:1950] T 5b124eda66364b7a81426934b98bca48 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:56.735780 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1001dd888ecd4166ba217506739f71e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.739810 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4258df74ea0841449846cdf1f6e00b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.740952 1181 tablet_bootstrap.cc:492] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:56.741089 1181 ts_tablet_manager.cc:1403] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.002s
I20260505 07:53:56.741897 1181 raft_consensus.cc:359] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.742060 1181 raft_consensus.cc:385] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.742141 1181 raft_consensus.cc:740] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:56.742365 1181 consensus_queue.cc:260] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.743886 1181 ts_tablet_manager.cc:1434] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260505 07:53:56.745081 1181 tablet_bootstrap.cc:492] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:56.746805 1181 tablet_bootstrap.cc:654] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.752398 761 tablet_service.cc:1511] Processing CreateTablet for tablet d37dc6df9da5401384a6fc5de868b41c (DEFAULT_TABLE table=default.Table_6c975d29ebde41ecb90fdaf8c72b2997 [id=0a01d809c30041a0a5b803341584f3ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.752873 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d37dc6df9da5401384a6fc5de868b41c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.752934 1172 tablet_bootstrap.cc:654] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.754427 749 tablet_service.cc:1511] Processing CreateTablet for tablet 08c6ed755fc64080a2310fe9a4e0a68b (DEFAULT_TABLE table=default.Table_a24821d9d3f14fee9ce992364d12721e [id=0b202734a7744102a98cc55ac980603b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.754694 758 tablet_service.cc:1558] Processing DeleteTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.755427 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 222c41e6cd8b4d0093fd692618efb107 with delete_type TABLET_DATA_DELETED (replaced by tablet b404e50ae092419c9396772b3b14eeaa) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.755820 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08c6ed755fc64080a2310fe9a4e0a68b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.755613 1178 ts_tablet_manager.cc:1929] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:56.757117 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de9e0fe8eb7477ab1cfa64b808575d8" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:53:56.756731 1368 leader_election.cc:290] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:56.757197 1178 log.cc:1199] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/b7cce14283e7405690a15e26be8a45b3
I20260505 07:53:56.759575 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 17327cee69a24e0cb493ff55f948bf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.760867 1181 tablet_bootstrap.cc:492] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:56.761124 1181 ts_tablet_manager.cc:1403] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.003s
W20260505 07:53:56.761945 717 leader_election.cc:343] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:56.762427 1181 raft_consensus.cc:359] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.762797 1181 raft_consensus.cc:385] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.762957 1181 raft_consensus.cc:740] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:56.763305 1181 consensus_queue.cc:260] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.764034 1181 ts_tablet_manager.cc:1434] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:53:56.764437 1181 tablet_bootstrap.cc:492] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:56.760723 1178 ts_tablet_manager.cc:1950] T b7cce14283e7405690a15e26be8a45b3 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:56.766275 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e707143a4e64725aa66db05b24999a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:56.770109 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet b7cce14283e7405690a15e26be8a45b3 (table default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]) successfully deleted
I20260505 07:53:56.771075 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f1feb0387bc4afead72346398dbe9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:56.771500 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 0baf83b155494fddb3c25144608706f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0baf83b155494fddb3c25144608706f7 already in progress: creating tablet
I20260505 07:53:56.769851 754 tablet_service.cc:1511] Processing CreateTablet for tablet 5191f9a357de4607aab6c7a8b4bcccdb (DEFAULT_TABLE table=default.Table_94e6f1ee4ab947d8bc9864ab0c8bd524 [id=f7b2051b32e240c1856be3f0802b2f3e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:56.773113 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet d8c9eaaef250461ca7a37d0cf42b07ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8c9eaaef250461ca7a37d0cf42b07ee already in progress: creating tablet
I20260505 07:53:56.774024 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 9862b87d74974dc49d7dc9049b2a694a (table default.Table_d1ecbab8fbf047cf847ebbb4aeee4b63 [id=164a806dd1a9450ea55b4ea1be0caa64]) successfully deleted
W20260505 07:53:56.774864 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 7e707143a4e64725aa66db05b24999a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e707143a4e64725aa66db05b24999a6 already in progress: creating tablet
W20260505 07:53:56.775389 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 0baf83b155494fddb3c25144608706f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0baf83b155494fddb3c25144608706f7 already in progress: creating tablet
W20260505 07:53:56.776033 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 3f1feb0387bc4afead72346398dbe9e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f1feb0387bc4afead72346398dbe9e5 already in progress: creating tablet
W20260505 07:53:56.699553 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 6429919aba17447dbdb393176360bbb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6429919aba17447dbdb393176360bbb0 already in progress: creating tablet
W20260505 07:53:56.776907 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 7e707143a4e64725aa66db05b24999a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e707143a4e64725aa66db05b24999a6 already in progress: creating tablet
W20260505 07:53:56.777184 585 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 77641.8us
W20260505 07:53:56.777256 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 17327cee69a24e0cb493ff55f948bf6c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 17327cee69a24e0cb493ff55f948bf6c already in progress: creating tablet
W20260505 07:53:56.777431 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 32c008bd99224a7595cecc607e5496a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32c008bd99224a7595cecc607e5496a9 already in progress: creating tablet
W20260505 07:53:56.777639 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 976359601ad74b149696dc7f555b550c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 976359601ad74b149696dc7f555b550c already in progress: creating tablet
I20260505 07:53:56.777752 1181 tablet_bootstrap.cc:654] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:56.777921 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet d8c9eaaef250461ca7a37d0cf42b07ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8c9eaaef250461ca7a37d0cf42b07ee already in progress: creating tablet
W20260505 07:53:56.778884 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 3f1feb0387bc4afead72346398dbe9e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f1feb0387bc4afead72346398dbe9e5 already in progress: creating tablet
W20260505 07:53:56.779942 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 17327cee69a24e0cb493ff55f948bf6c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 17327cee69a24e0cb493ff55f948bf6c already in progress: creating tablet
I20260505 07:53:56.780361 1175 tablet_replica.cc:333] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
W20260505 07:53:56.781459 586 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 571257bbb7e94315b60d84d6aa4917b1 (table default.Table_087f86f68d1a411a8aba986365a3b2fe [id=6cb8bc8043474bad9f4278b7eef7c71a], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 571257bbb7e94315b60d84d6aa4917b1 consensus state has no leader
I20260505 07:53:56.783972 1172 tablet_bootstrap.cc:492] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:56.784173 1172 ts_tablet_manager.cc:1403] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.067s user 0.000s sys 0.008s
I20260505 07:53:56.785046 1172 raft_consensus.cc:359] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.785269 1172 raft_consensus.cc:385] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.785355 1172 raft_consensus.cc:740] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:56.785642 1172 consensus_queue.cc:260] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.786139 758 tablet_service.cc:1511] Processing CreateTablet for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (DEFAULT_TABLE table=default.Table_cb21f88a808c477b8a55a83e7507b951 [id=2c06ff8eed31432e87f08e9b59049652]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.786959 746 tablet_service.cc:1511] Processing CreateTablet for tablet 8eb7eaff8837485a8ae3ccb44c60471d (DEFAULT_TABLE table=default.Table_3e9abeb1ce814ef2840df1ce519f1768 [id=266ff796c7a74c23816dd9c422e9d005]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.787079 1172 ts_tablet_manager.cc:1434] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:56.788092 1172 tablet_bootstrap.cc:492] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:56.789832 1172 tablet_bootstrap.cc:654] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.792136 1175 raft_consensus.cc:2243] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:56.792428 1175 raft_consensus.cc:2272] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:56.794049 1181 tablet_bootstrap.cc:492] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:56.794314 1181 ts_tablet_manager.cc:1403] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.009s
I20260505 07:53:56.795684 1181 raft_consensus.cc:359] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:53:56.777884 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet ccdde3e5a13e436d9f2328831e155adb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ccdde3e5a13e436d9f2328831e155adb already in progress: creating tablet
I20260505 07:53:56.795977 1181 raft_consensus.cc:385] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.796142 1181 raft_consensus.cc:740] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
W20260505 07:53:56.796481 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 6da10815e0dc41a1a07b6786882fffd3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6da10815e0dc41a1a07b6786882fffd3 already in progress: creating tablet
I20260505 07:53:56.796494 1181 consensus_queue.cc:260] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:53:56.798257 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 78a6470720984761a8606abeafbef992 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78a6470720984761a8606abeafbef992 already in progress: creating tablet
W20260505 07:53:56.798580 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 4a79578757824bfc9df129e3475e7af3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4a79578757824bfc9df129e3475e7af3 already in progress: creating tablet
I20260505 07:53:56.798430 1181 ts_tablet_manager.cc:1434] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260505 07:53:56.799389 1172 tablet_bootstrap.cc:492] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:56.799639 1172 ts_tablet_manager.cc:1403] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.001s
I20260505 07:53:56.799707 1181 tablet_bootstrap.cc:492] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:56.800405 1172 raft_consensus.cc:359] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.800594 1172 raft_consensus.cc:385] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.800675 1172 raft_consensus.cc:740] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:56.800922 1172 consensus_queue.cc:260] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.802591 1181 tablet_bootstrap.cc:654] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.802623 1172 ts_tablet_manager.cc:1434] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:56.802910 1172 tablet_bootstrap.cc:492] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:56.804382 1175 ts_tablet_manager.cc:1916] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:56.807564 1172 tablet_bootstrap.cc:654] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:56.807919 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de9e0fe8eb7477ab1cfa64b808575d8" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:53:56.808270 921 raft_consensus.cc:3060] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:56.814118 921 raft_consensus.cc:2468] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:53:56.815336 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5191f9a357de4607aab6c7a8b4bcccdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.816506 1364 raft_consensus.cc:493] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:56.816709 1364 raft_consensus.cc:515] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.817755 1364 leader_election.cc:290] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:53:56.819044 991 leader_election.cc:343] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 09a8dbb7fe4b406d804c0b62ba006adb
I20260505 07:53:56.819164 760 tablet_service.cc:1511] Processing CreateTablet for tablet 4258df74ea0841449846cdf1f6e00b9b (DEFAULT_TABLE table=default.Table_b3b9364eea524724b176de87f62eb49e [id=1fd77c8cfe9d40dab5da490774294796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.832137 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 5b124eda66364b7a81426934b98bca48 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]) successfully deleted
W20260505 07:53:56.833640 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 1838f89dc14a47f99f95f4cc5fdefd70 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1838f89dc14a47f99f95f4cc5fdefd70 already in progress: creating tablet
I20260505 07:53:56.840975 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4258df74ea0841449846cdf1f6e00b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.843746 750 tablet_service.cc:1511] Processing CreateTablet for tablet d3d11bd19f074bbab101448b45426b7a (DEFAULT_TABLE table=default.Table_cf18ef8e6a56482883dba2744997604a [id=b131fb4265684cd2b6506a7abeb29871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.844339 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3d11bd19f074bbab101448b45426b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.851917 718 leader_election.cc:304] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
W20260505 07:53:56.852885 988 leader_election.cc:343] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 09a8dbb7fe4b406d804c0b62ba006adb
I20260505 07:53:56.853094 988 leader_election.cc:304] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:56.853641 1364 raft_consensus.cc:2749] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:56.852897 762 tablet_service.cc:1511] Processing CreateTablet for tablet 1001dd888ecd4166ba217506739f71e8 (DEFAULT_TABLE table=default.Table_09561894665e45a98f269fd1d764b78a [id=048ca74dfef1481d945932e17b785ce3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:56.855968 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1001dd888ecd4166ba217506739f71e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.861951 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62bf3a6bc0ac42a99a4181f820e3ca87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.862784 1383 tablet.cc:1724] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:53:56.864527 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8eb7eaff8837485a8ae3ccb44c60471d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:56.871639 1379 raft_consensus.cc:2804] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:56.874845 1370 mvcc.cc:204] Tried to move back new op lower bound from 7282555438583570432 to 7282555435750510592. Current Snapshot: MvccSnapshot[applied={T|T < 7282555438583570432}]
I20260505 07:53:56.885200 1176 tablet_bootstrap.cc:492] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:56.886168 1176 ts_tablet_manager.cc:1403] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.271s user 0.004s sys 0.007s
W20260505 07:53:56.837324 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 1852f7f4ff594639908b03730b68fe86 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1852f7f4ff594639908b03730b68fe86 already in progress: creating tablet
W20260505 07:53:56.912199 585 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 75444.1us
W20260505 07:53:56.912603 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 003cd4472b424aacbba8c5f89900e228 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 003cd4472b424aacbba8c5f89900e228 already in progress: creating tablet
W20260505 07:53:56.913065 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 1ef4f516e64a44fd8cd33674fb808067 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1ef4f516e64a44fd8cd33674fb808067 already in progress: creating tablet
W20260505 07:53:56.913357 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 93d878ae0ddb42319560f74e0a490321 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 93d878ae0ddb42319560f74e0a490321 already in progress: creating tablet
W20260505 07:53:56.913738 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 7585f00119e14006990ed551667b4486 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7585f00119e14006990ed551667b4486 already in progress: creating tablet
I20260505 07:53:56.927568 1176 raft_consensus.cc:359] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.928481 1389 tablet.cc:1724] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:53:56.931207 1176 raft_consensus.cc:385] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:56.931411 1176 raft_consensus.cc:740] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
W20260505 07:53:56.941236 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 06904a54b1c5463da31af0bc00fb606f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 06904a54b1c5463da31af0bc00fb606f already in progress: creating tablet
W20260505 07:53:56.941627 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 0baf83b155494fddb3c25144608706f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0baf83b155494fddb3c25144608706f7 already in progress: creating tablet
W20260505 07:53:56.941967 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet d8c9eaaef250461ca7a37d0cf42b07ee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d8c9eaaef250461ca7a37d0cf42b07ee already in progress: creating tablet
I20260505 07:53:56.946169 1366 raft_consensus.cc:493] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:56.946327 1366 raft_consensus.cc:515] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.947045 1366 leader_election.cc:290] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:56.947866 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faf1140fe05e4f9493563ab8464c582b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:56.949198 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faf1140fe05e4f9493563ab8464c582b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:56.949621 856 leader_election.cc:343] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:53:56.950014 854 leader_election.cc:343] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:56.950150 854 leader_election.cc:304] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:56.950539 1366 raft_consensus.cc:2749] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:56.955241 1176 consensus_queue.cc:260] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.960501 1356 mvcc.cc:204] Tried to move back new op lower bound from 7282555438583570432 to 7282555435750510592. Current Snapshot: MvccSnapshot[applied={T|T < 7282555438583570432}]
I20260505 07:53:56.966536 1176 ts_tablet_manager.cc:1434] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.071s user 0.003s sys 0.000s
I20260505 07:53:56.968444 1176 tablet_bootstrap.cc:492] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:56.969250 1383 tablet.cc:1724] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260505 07:53:56.987746 1379 raft_consensus.cc:697] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:53:56.988231 1379 consensus_queue.cc:237] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:56.994055 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_75872f5895d34e0ab0677a846b5b5926"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:56.994916 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_75872f5895d34e0ab0677a846b5b5926 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:56.995148 1181 tablet_bootstrap.cc:492] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:56.995697 1181 ts_tablet_manager.cc:1403] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.196s user 0.023s sys 0.122s
I20260505 07:53:56.998229 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d0fa020605d347b3ad7096aed39c552c" } new_table_name: "default.Table_d533362919954c83aa119d8607d882d6" modify_external_catalogs: true
I20260505 07:53:56.999979 601 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_27c52b9b060d4dd5a80799a987d5b7c9" } modify_external_catalogs: true
I20260505 07:53:57.000164 601 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_27c52b9b060d4dd5a80799a987d5b7c9" } modify_external_catalogs: true
I20260505 07:53:57.002384 1172 tablet_bootstrap.cc:492] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:57.002591 1172 ts_tablet_manager.cc:1403] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.200s user 0.094s sys 0.100s
I20260505 07:53:57.002825 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_911918ab5f5145178d2490fb80e7db21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.003250 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_911918ab5f5145178d2490fb80e7db21 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.004789 1175 ts_tablet_manager.cc:1929] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.004961 1175 log.cc:1199] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/222c41e6cd8b4d0093fd692618efb107
I20260505 07:53:57.005194 1176 tablet_bootstrap.cc:654] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.005151 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f3fff69ccbc54a58bdad7a8b5489d525"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.005638 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3fff69ccbc54a58bdad7a8b5489d525 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.006860 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7d1747ea770445058549830f424a7aa3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.007272 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d1747ea770445058549830f424a7aa3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.008494 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_eb5ee86732cf4963809c3838e7654571" } new_table_name: "default.Table_45faf5ec25414b7e8126ba77d62ca031" modify_external_catalogs: true
I20260505 07:53:57.008801 1172 raft_consensus.cc:359] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.009073 1172 raft_consensus.cc:385] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.009157 1172 raft_consensus.cc:740] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.009456 1172 consensus_queue.cc:260] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.010128 1172 ts_tablet_manager.cc:1434] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260505 07:53:57.010143 593 catalog_manager.cc:5671] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } }
I20260505 07:53:57.010433 597 master_service.cc:954] ReplaceTablet: received request to replace tablet cac5064f68d24ebe95b5a0b8e10f463e from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.010550 593 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e]). Expected version 2 got 0
W20260505 07:53:57.011978 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e278b86e7b5d444ebad8d0f6921d4c62 consensus state has no leader
I20260505 07:53:57.012392 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 6429919aba17447dbdb393176360bbb0 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.012935 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 12501fdf2cc5491cb1a06db490e70810 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.013996 596 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2071445d362745d9b2bbc7712d032c17" } new_table_name: "default.Table_5fd319488e1d4b9da8f13212d678a934" modify_external_catalogs: true
I20260505 07:53:57.016000 594 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:53:57.016624 1175 ts_tablet_manager.cc:1950] T 222c41e6cd8b4d0093fd692618efb107 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:57.019785 1172 tablet_bootstrap.cc:492] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:57.022100 1181 raft_consensus.cc:359] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.022360 1181 raft_consensus.cc:385] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.022452 1181 raft_consensus.cc:740] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.022714 1389 tablet.cc:1724] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260505 07:53:57.023061 1181 consensus_queue.cc:260] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.022897 1172 tablet_bootstrap.cc:654] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.024827 1181 ts_tablet_manager.cc:1434] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.029s user 0.003s sys 0.000s
I20260505 07:53:57.025421 1181 tablet_bootstrap.cc:492] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.031395 1181 tablet_bootstrap.cc:654] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.034251 601 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 4ba2cc218300472cb5fd6dc8cfb7811c
I20260505 07:53:57.035297 601 master_service.cc:954] ReplaceTablet: received request to replace tablet 8b57c1c618484d5d86a846fb982a2240 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.036093 1181 tablet_bootstrap.cc:492] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.036134 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.036201 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.036402 1181 ts_tablet_manager.cc:1403] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
W20260505 07:53:57.036731 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 4ba2cc218300472cb5fd6dc8cfb7811c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ba2cc218300472cb5fd6dc8cfb7811c already in progress: creating tablet
W20260505 07:53:57.037209 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 4ba2cc218300472cb5fd6dc8cfb7811c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ba2cc218300472cb5fd6dc8cfb7811c already in progress: creating tablet
I20260505 07:53:57.037364 594 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:53:57.037508 1181 raft_consensus.cc:359] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.037672 1181 raft_consensus.cc:385] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.037768 1181 raft_consensus.cc:740] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.038049 1181 consensus_queue.cc:260] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.038451 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_fd75ee1f117344169ccafd58dc18408f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.039458 1181 ts_tablet_manager.cc:1434] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260505 07:53:57.039975 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd75ee1f117344169ccafd58dc18408f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.041330 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 222c41e6cd8b4d0093fd692618efb107 (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]) successfully deleted
I20260505 07:53:57.041977 1181 tablet_bootstrap.cc:492] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:53:57.041627 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 17327cee69a24e0cb493ff55f948bf6c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 17327cee69a24e0cb493ff55f948bf6c already in progress: creating tablet
W20260505 07:53:57.043004 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 7e707143a4e64725aa66db05b24999a6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7e707143a4e64725aa66db05b24999a6 already in progress: creating tablet
I20260505 07:53:57.043432 1181 tablet_bootstrap.cc:654] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:57.043985 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 3f1feb0387bc4afead72346398dbe9e5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3f1feb0387bc4afead72346398dbe9e5 already in progress: creating tablet
I20260505 07:53:57.045565 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.046829 1181 tablet_bootstrap.cc:492] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.047020 1181 ts_tablet_manager.cc:1403] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260505 07:53:57.048017 1181 raft_consensus.cc:359] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.048291 1181 raft_consensus.cc:385] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.048365 1181 raft_consensus.cc:740] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.048583 1181 consensus_queue.cc:260] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.049966 1181 ts_tablet_manager.cc:1434] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:57.050992 1181 tablet_bootstrap.cc:492] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.053218 1181 tablet_bootstrap.cc:654] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:57.053812 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5683d7366de84653ace56f0edcd3818c consensus state has no leader
I20260505 07:53:57.054867 598 master_service.cc:954] ReplaceTablet: received request to replace tablet 3d6494590052459485059f5dce46f34f from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.060074 1181 tablet_bootstrap.cc:492] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.060216 1181 ts_tablet_manager.cc:1403] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260505 07:53:57.061021 1181 raft_consensus.cc:359] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.061188 1181 raft_consensus.cc:385] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.061275 1181 raft_consensus.cc:740] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.061482 1181 consensus_queue.cc:260] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.062378 1366 raft_consensus.cc:493] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.062562 1366 raft_consensus.cc:515] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.062920 1181 ts_tablet_manager.cc:1434] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:57.063216 1366 leader_election.cc:290] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.063808 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3272555a8c814c0ebe61ca32d715212b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:57.064006 1181 tablet_bootstrap.cc:492] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:53:57.064337 856 leader_election.cc:343] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.064435 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3272555a8c814c0ebe61ca32d715212b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:57.065069 854 leader_election.cc:343] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.065215 854 leader_election.cc:304] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.065478 1366 raft_consensus.cc:2749] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.066079 1181 tablet_bootstrap.cc:654] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.068236 1181 tablet_bootstrap.cc:492] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.068379 1181 ts_tablet_manager.cc:1403] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260505 07:53:57.069084 1181 raft_consensus.cc:359] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.069265 1181 raft_consensus.cc:385] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.069340 1181 raft_consensus.cc:740] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.069559 1181 consensus_queue.cc:260] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.071065 1181 ts_tablet_manager.cc:1434] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:53:57.072221 1181 tablet_bootstrap.cc:492] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.074338 1181 tablet_bootstrap.cc:654] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.082998 1181 tablet_bootstrap.cc:492] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.083135 1181 ts_tablet_manager.cc:1403] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.009s
I20260505 07:53:57.083956 1181 raft_consensus.cc:359] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.084126 1181 raft_consensus.cc:385] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.084220 1181 raft_consensus.cc:740] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.084448 1181 consensus_queue.cc:260] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.085639 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet cac5064f68d24ebe95b5a0b8e10f463e
I20260505 07:53:57.085892 1181 ts_tablet_manager.cc:1434] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:57.086184 597 catalog_manager.cc:6741] ReplaceTablet: tablet cac5064f68d24ebe95b5a0b8e10f463e deleted and replaced by tablet dd3fc8a9d9f04636b1f31bb88411c1da
I20260505 07:53:57.087057 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e with delete_type TABLET_DATA_DELETED (replaced by tablet dd3fc8a9d9f04636b1f31bb88411c1da) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.087200 891 tablet_service.cc:1558] Processing DeleteTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e with delete_type TABLET_DATA_DELETED (replaced by tablet dd3fc8a9d9f04636b1f31bb88411c1da) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:57.087905 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet cac5064f68d24ebe95b5a0b8e10f463e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cac5064f68d24ebe95b5a0b8e10f463e already in progress: creating tablet
I20260505 07:53:57.087952 1181 tablet_bootstrap.cc:492] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:53:57.088253 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet cac5064f68d24ebe95b5a0b8e10f463e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cac5064f68d24ebe95b5a0b8e10f463e already in progress: creating tablet
I20260505 07:53:57.089476 1181 tablet_bootstrap.cc:654] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.089715 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 12501fdf2cc5491cb1a06db490e70810
I20260505 07:53:57.090211 592 catalog_manager.cc:6741] ReplaceTablet: tablet 12501fdf2cc5491cb1a06db490e70810 deleted and replaced by tablet d622612bdc404727a27b83769bf39738
I20260505 07:53:57.091409 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 12501fdf2cc5491cb1a06db490e70810 with delete_type TABLET_DATA_DELETED (replaced by tablet d622612bdc404727a27b83769bf39738) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.092111 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 12501fdf2cc5491cb1a06db490e70810 with delete_type TABLET_DATA_DELETED (replaced by tablet d622612bdc404727a27b83769bf39738) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:57.093595 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 12501fdf2cc5491cb1a06db490e70810 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12501fdf2cc5491cb1a06db490e70810 already in progress: creating tablet
W20260505 07:53:57.093959 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 12501fdf2cc5491cb1a06db490e70810 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12501fdf2cc5491cb1a06db490e70810 already in progress: creating tablet
I20260505 07:53:57.095754 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_fd8baf98a0f64d7fbd3d61d32af60991"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.096556 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fd8baf98a0f64d7fbd3d61d32af60991 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.099179 1181 tablet_bootstrap.cc:492] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.099331 1181 ts_tablet_manager.cc:1403] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260505 07:53:57.100179 1181 raft_consensus.cc:359] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.100718 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_0b3a73b640a644579fa864cce1a7ecbe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.101415 1181 raft_consensus.cc:385] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.101500 1181 raft_consensus.cc:740] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
W20260505 07:53:57.101701 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b3a73b640a644579fa864cce1a7ecbe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.101752 1181 consensus_queue.cc:260] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.103044 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_db54454473d84a83b6c77d06a1f09618"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.103467 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db54454473d84a83b6c77d06a1f09618 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.103884 1181 ts_tablet_manager.cc:1434] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:53:57.104774 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_614ea89c84d342f0a1c4912e8e77fe47"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.105031 1181 tablet_bootstrap.cc:492] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:53:57.105193 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_614ea89c84d342f0a1c4912e8e77fe47 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.106701 1181 tablet_bootstrap.cc:654] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.109640 1181 tablet_bootstrap.cc:492] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.109763 1181 ts_tablet_manager.cc:1403] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260505 07:53:57.110080 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4ddf6f25bf834287b5808ee1c64474bd" } new_table_name: "default.Table_3ad26a52aa5e4b0692fb55aa761970ed" modify_external_catalogs: true
I20260505 07:53:57.110525 1181 raft_consensus.cc:359] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.110675 1181 raft_consensus.cc:385] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.110775 1181 raft_consensus.cc:740] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.111022 1181 consensus_queue.cc:260] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.112407 1181 ts_tablet_manager.cc:1434] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
W20260505 07:53:57.112903 596 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 83e62b7af9b847b7a9005e4345905c0d consensus state has no leader
I20260505 07:53:57.113420 1181 tablet_bootstrap.cc:492] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.114327 601 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 8b57c1c618484d5d86a846fb982a2240
I20260505 07:53:57.114938 1181 tablet_bootstrap.cc:654] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.115212 601 catalog_manager.cc:6741] ReplaceTablet: tablet 8b57c1c618484d5d86a846fb982a2240 deleted and replaced by tablet de1a667b14fe494c8ee4be0f47a340be
I20260505 07:53:57.116364 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b57c1c618484d5d86a846fb982a2240 with delete_type TABLET_DATA_DELETED (replaced by tablet de1a667b14fe494c8ee4be0f47a340be) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:57.117336 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 8b57c1c618484d5d86a846fb982a2240 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b57c1c618484d5d86a846fb982a2240 already in progress: creating tablet
I20260505 07:53:57.117451 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b57c1c618484d5d86a846fb982a2240 with delete_type TABLET_DATA_DELETED (replaced by tablet de1a667b14fe494c8ee4be0f47a340be) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:57.118010 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8b57c1c618484d5d86a846fb982a2240 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b57c1c618484d5d86a846fb982a2240 already in progress: creating tablet
I20260505 07:53:57.119243 1181 tablet_bootstrap.cc:492] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.119217 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_43b383b7a58b493c8f5b2d9f3038a2e2" } modify_external_catalogs: true
I20260505 07:53:57.119387 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_43b383b7a58b493c8f5b2d9f3038a2e2" } modify_external_catalogs: true
I20260505 07:53:57.119465 1181 ts_tablet_manager.cc:1403] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260505 07:53:57.119760 1364 raft_consensus.cc:493] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.119902 1364 raft_consensus.cc:515] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.120471 1364 leader_election.cc:290] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.120868 601 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b3b9364eea524724b176de87f62eb49e" } modify_external_catalogs: true
I20260505 07:53:57.121023 601 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b3b9364eea524724b176de87f62eb49e" } modify_external_catalogs: true
I20260505 07:53:57.120975 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ec347e754ab4b7282e68b15a9bbf993" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:57.121491 988 leader_election.cc:343] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.121491 1181 raft_consensus.cc:359] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.121855 1181 raft_consensus.cc:385] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.121977 1181 raft_consensus.cc:740] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.122256 1181 consensus_queue.cc:260] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:57.122418 991 leader_election.cc:343] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 2ec347e754ab4b7282e68b15a9bbf993
I20260505 07:53:57.122591 991 leader_election.cc:304] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.123664 1364 raft_consensus.cc:2749] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.124208 1181 ts_tablet_manager.cc:1434] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:53:57.125402 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_82c755fca0b64337a44a63876e04fa68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.126024 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82c755fca0b64337a44a63876e04fa68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.126725 1181 tablet_bootstrap.cc:492] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.128510 1181 tablet_bootstrap.cc:654] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:57.130165 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 4ba2cc218300472cb5fd6dc8cfb7811c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ba2cc218300472cb5fd6dc8cfb7811c already in progress: creating tablet
I20260505 07:53:57.140475 1181 tablet_bootstrap.cc:492] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.140492 758 tablet_service.cc:1558] Processing DeleteTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e with delete_type TABLET_DATA_DELETED (replaced by tablet dd3fc8a9d9f04636b1f31bb88411c1da) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.140501 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 12501fdf2cc5491cb1a06db490e70810 with delete_type TABLET_DATA_DELETED (replaced by tablet d622612bdc404727a27b83769bf39738) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.140614 1181 ts_tablet_manager.cc:1403] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.010s
W20260505 07:53:57.141341 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet cac5064f68d24ebe95b5a0b8e10f463e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cac5064f68d24ebe95b5a0b8e10f463e already in progress: creating tablet
I20260505 07:53:57.141336 1181 raft_consensus.cc:359] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.141502 1181 raft_consensus.cc:385] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.141587 1181 raft_consensus.cc:740] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.141613 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b57c1c618484d5d86a846fb982a2240 with delete_type TABLET_DATA_DELETED (replaced by tablet de1a667b14fe494c8ee4be0f47a340be) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:57.141628 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 12501fdf2cc5491cb1a06db490e70810 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 12501fdf2cc5491cb1a06db490e70810 already in progress: creating tablet
I20260505 07:53:57.147742 1181 consensus_queue.cc:260] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.148265 1181 ts_tablet_manager.cc:1434] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260505 07:53:57.148557 1181 tablet_bootstrap.cc:492] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.148815 595 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52] alter complete (version 4)
W20260505 07:53:57.149248 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 8b57c1c618484d5d86a846fb982a2240 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b57c1c618484d5d86a846fb982a2240 already in progress: creating tablet
I20260505 07:53:57.150750 1181 tablet_bootstrap.cc:654] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.151902 754 tablet_service.cc:1558] Processing DeleteTablet for tablet cf95f4331e4540ae84e329cf577427b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.152124 1175 tablet_replica.cc:333] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:57.152275 1175 raft_consensus.cc:2243] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.152437 1175 raft_consensus.cc:2272] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.153064 1175 ts_tablet_manager.cc:1916] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.153057 899 tablet_service.cc:1558] Processing DeleteTablet for tablet e85b44337f62494c997c0008fc492aab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.153297 883 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.153439 897 tablet_service.cc:1558] Processing DeleteTablet for tablet faf1140fe05e4f9493563ab8464c582b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.153618 1171 tablet_replica.cc:333] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:57.153797 1171 raft_consensus.cc:2243] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.153932 1171 raft_consensus.cc:2272] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.154458 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 3272555a8c814c0ebe61ca32d715212b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.155067 1171 ts_tablet_manager.cc:1916] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.161232 1175 ts_tablet_manager.cc:1929] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.161396 1175 log.cc:1199] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/cf95f4331e4540ae84e329cf577427b8
I20260505 07:53:57.161865 1175 ts_tablet_manager.cc:1950] T cf95f4331e4540ae84e329cf577427b8 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:57.163152 1176 tablet_bootstrap.cc:492] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:57.163305 1176 ts_tablet_manager.cc:1403] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.195s user 0.014s sys 0.013s
I20260505 07:53:57.164907 1176 raft_consensus.cc:359] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.165601 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet cf95f4331e4540ae84e329cf577427b8 (table default.Table_8a35dff4878b4429b4a6600b8259fd83 [id=0e331c73bbf842dcac6c971a6688f58b]) successfully deleted
I20260505 07:53:57.172205 1171 ts_tablet_manager.cc:1929] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.172581 1171 log.cc:1199] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e85b44337f62494c997c0008fc492aab
I20260505 07:53:57.173061 1171 ts_tablet_manager.cc:1950] T e85b44337f62494c997c0008fc492aab P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:57.173746 1176 raft_consensus.cc:385] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.174192 1176 raft_consensus.cc:740] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.174762 1176 consensus_queue.cc:260] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.175196 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e85b44337f62494c997c0008fc492aab (table default.Table_b8745d36d43a4f42a5b3d43a37c21e55 [id=06480a024264477596ab9058373c273b]) successfully deleted
I20260505 07:53:57.175803 1171 tablet_replica.cc:333] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:57.177531 1176 ts_tablet_manager.cc:1434] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.014s user 0.003s sys 0.000s
I20260505 07:53:57.178681 1181 tablet_bootstrap.cc:492] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.178865 1181 ts_tablet_manager.cc:1403] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.030s user 0.017s sys 0.009s
I20260505 07:53:57.179031 1176 tablet_bootstrap.cc:492] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:57.180357 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 43e4c92b7c3140fdb2d6a290aaf0ad55
I20260505 07:53:57.180971 1172 tablet_bootstrap.cc:492] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:57.181097 1172 ts_tablet_manager.cc:1403] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.161s user 0.005s sys 0.030s
I20260505 07:53:57.181133 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.181754 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.181821 1172 raft_consensus.cc:359] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:57.181917 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 already in progress: creating tablet
I20260505 07:53:57.181972 1172 raft_consensus.cc:385] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.182062 1172 raft_consensus.cc:740] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.180716 1176 tablet_bootstrap.cc:654] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.182281 1172 consensus_queue.cc:260] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:57.182436 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 already in progress: creating tablet
I20260505 07:53:57.182693 1172 ts_tablet_manager.cc:1434] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:53:57.179641 1181 raft_consensus.cc:359] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.183161 1181 raft_consensus.cc:385] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.183245 1181 raft_consensus.cc:740] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.183495 1181 consensus_queue.cc:260] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.184264 1181 ts_tablet_manager.cc:1434] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:53:57.184599 1181 tablet_bootstrap.cc:492] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.185113 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:57.187413 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 909306e1e5f34b7c8fbc0d86cafce531 consensus state has no leader
I20260505 07:53:57.189046 1172 tablet_bootstrap.cc:492] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:57.190852 1181 tablet_bootstrap.cc:654] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.192500 1172 tablet_bootstrap.cc:654] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.192710 601 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 4258df74ea0841449846cdf1f6e00b9b
I20260505 07:53:57.193354 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 4258df74ea0841449846cdf1f6e00b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:57.194573 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 4258df74ea0841449846cdf1f6e00b9b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4258df74ea0841449846cdf1f6e00b9b already in progress: creating tablet
I20260505 07:53:57.194662 1176 tablet_bootstrap.cc:492] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:57.194825 1176 ts_tablet_manager.cc:1403] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.002s
I20260505 07:53:57.196141 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 4258df74ea0841449846cdf1f6e00b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.196656 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 4258df74ea0841449846cdf1f6e00b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:57.196770 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 4258df74ea0841449846cdf1f6e00b9b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4258df74ea0841449846cdf1f6e00b9b already in progress: creating tablet
I20260505 07:53:57.195673 1176 raft_consensus.cc:359] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.197585 1176 raft_consensus.cc:385] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.197715 1176 raft_consensus.cc:740] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.198014 1176 consensus_queue.cc:260] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.199903 593 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:53:57.199940 1176 ts_tablet_manager.cc:1434] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:53:57.201687 1176 tablet_bootstrap.cc:492] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:57.203244 1176 tablet_bootstrap.cc:654] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.204245 1181 tablet_bootstrap.cc:492] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.204464 1181 ts_tablet_manager.cc:1403] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260505 07:53:57.205313 1181 raft_consensus.cc:359] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:57.205550 1181 raft_consensus.cc:385] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.205647 1181 raft_consensus.cc:740] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.205873 1181 consensus_queue.cc:260] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:57.206207 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0562bf620b5f46ecab060957ac1ca8bf" } modify_external_catalogs: true
I20260505 07:53:57.206283 1181 ts_tablet_manager.cc:1434] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:53:57.206362 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0562bf620b5f46ecab060957ac1ca8bf" } modify_external_catalogs: true
I20260505 07:53:57.206537 1181 tablet_bootstrap.cc:492] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.210201 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_da224a28c7e8451a8f4c04fcd2160c6b" } new_table_name: "default.Table_c41ba7470a784645bdddd3a262d4bb4e" modify_external_catalogs: true
I20260505 07:53:57.210644 1181 tablet_bootstrap.cc:654] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.210142 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6490f568024510ab843ec5205a4d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.212604 1175 tablet_replica.cc:333] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:57.213652 1181 tablet_bootstrap.cc:492] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.213809 1181 ts_tablet_manager.cc:1403] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260505 07:53:57.213829 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2f6f14275b8847be96f68a761156a854" } modify_external_catalogs: true
I20260505 07:53:57.214046 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2f6f14275b8847be96f68a761156a854" } modify_external_catalogs: true
I20260505 07:53:57.214298 1175 raft_consensus.cc:2243] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.214480 1175 raft_consensus.cc:2272] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.214480 1181 raft_consensus.cc:359] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.214640 1181 raft_consensus.cc:385] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.214730 1181 raft_consensus.cc:740] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.214980 1181 consensus_queue.cc:260] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.215222 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_e9ed38d78117420d98a771675ab512bb" } new_table_name: "default.Table_563025047e4142d4807295cff836ea99" modify_external_catalogs: true
I20260505 07:53:57.215601 1181 ts_tablet_manager.cc:1434] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:53:57.215874 1181 tablet_bootstrap.cc:492] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.216997 1175 ts_tablet_manager.cc:1916] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.217443 1181 tablet_bootstrap.cc:654] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.222981 1364 raft_consensus.cc:493] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.223551 1364 raft_consensus.cc:515] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.224182 1364 leader_election.cc:290] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:57.224524 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 222c41e6cd8b4d0093fd692618efb107 with delete_type TABLET_DATA_DELETED (replaced by tablet b404e50ae092419c9396772b3b14eeaa) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.224781 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3781d0bbcd1b4189bd836eeae4236f40" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:53:57.224994 921 raft_consensus.cc:2468] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:53:57.225055 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet 772a65f4016843d18fbca25842f76de0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.225035 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4d96bdf674c2395938b9971c0ad28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e707143a4e64725aa66db05b24999a6) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.225400 991 leader_election.cc:304] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:53:57.226037 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3781d0bbcd1b4189bd836eeae4236f40" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:57.226444 1364 raft_consensus.cc:2804] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:57.227305 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet fea61fad19b34ffaa0b9df3a51979b18 with delete_type TABLET_DATA_DELETED (replaced by tablet e278b86e7b5d444ebad8d0f6921d4c62) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.228466 1364 raft_consensus.cc:493] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:57.228595 1364 raft_consensus.cc:3060] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:57.228830 1178 tablet_replica.cc:333] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:57.229000 1178 raft_consensus.cc:2243] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.229144 1178 raft_consensus.cc:2272] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.229794 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 09a8dbb7fe4b406d804c0b62ba006adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.230007 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 2850a301ac4f435a96742b00bf54baa9 with delete_type TABLET_DATA_DELETED (replaced by tablet 956b5e05bda648349ba26d5c1ea3b279) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:57.230180 988 leader_election.cc:343] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.230492 1175 ts_tablet_manager.cc:1929] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.230628 1175 log.cc:1199] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/4d6490f568024510ab843ec5205a4d1a
I20260505 07:53:57.231029 1175 ts_tablet_manager.cc:1950] T 4d6490f568024510ab843ec5205a4d1a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:57.232892 1364 raft_consensus.cc:515] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.233470 1364 leader_election.cc:290] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:57.233677 1171 raft_consensus.cc:2243] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.233940 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3781d0bbcd1b4189bd836eeae4236f40" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:53:57.233945 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3781d0bbcd1b4189bd836eeae4236f40" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:53:57.234093 1171 raft_consensus.cc:2272] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.234138 921 raft_consensus.cc:3060] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:53:57.234433 988 leader_election.cc:343] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.234655 1178 ts_tablet_manager.cc:1916] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.235024 1172 tablet_bootstrap.cc:492] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:57.235184 1172 ts_tablet_manager.cc:1403] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.052s user 0.009s sys 0.009s
I20260505 07:53:57.237046 1171 ts_tablet_manager.cc:1916] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.237633 921 raft_consensus.cc:2468] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:53:57.238145 991 leader_election.cc:304] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.240335 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4d96bdf674c2395938b9971c0ad28 with delete_type TABLET_DATA_DELETED (replaced by tablet 7e707143a4e64725aa66db05b24999a6) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.240229 1172 raft_consensus.cc:359] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.240453 1172 raft_consensus.cc:385] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.240530 1172 raft_consensus.cc:740] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.240649 1364 raft_consensus.cc:493] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.240765 1364 raft_consensus.cc:515] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.241665 1364 leader_election.cc:290] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.242082 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f95053c53c17434b9175534e6d3ac45e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.243237 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f95053c53c17434b9175534e6d3ac45e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:57.242609 988 leader_election.cc:343] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 772a65f4016843d18fbca25842f76de0
I20260505 07:53:57.243741 1364 raft_consensus.cc:2804] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:57.244014 1175 tablet_replica.cc:333] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:57.243345 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 4d6490f568024510ab843ec5205a4d1a (table default.Table_ae9bf3e293014ec781c8885d8a716e83 [id=9c204b77f3df4bf9a58508967c89c9f4]) successfully deleted
I20260505 07:53:57.244347 1364 raft_consensus.cc:697] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:53:57.244722 1175 raft_consensus.cc:2243] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.244938 1175 raft_consensus.cc:2272] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.246637 1364 consensus_queue.cc:237] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.247645 1172 consensus_queue.cc:260] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.247865 1175 ts_tablet_manager.cc:1916] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.248263 1172 ts_tablet_manager.cc:1434] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.013s user 0.003s sys 0.000s
I20260505 07:53:57.249395 1172 tablet_bootstrap.cc:492] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:57.252188 1176 tablet_bootstrap.cc:492] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:57.252418 1176 ts_tablet_manager.cc:1403] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.052s user 0.010s sys 0.000s
I20260505 07:53:57.253439 1176 raft_consensus.cc:359] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.253746 1176 raft_consensus.cc:385] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.253892 1176 raft_consensus.cc:740] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.254220 1176 consensus_queue.cc:260] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.256330 1176 ts_tablet_manager.cc:1434] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:53:57.258086 1176 tablet_bootstrap.cc:492] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:57.259482 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_82a545edb422426ab7eaeb4ae68a1a3f" } modify_external_catalogs: true
I20260505 07:53:57.259749 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_82a545edb422426ab7eaeb4ae68a1a3f" } modify_external_catalogs: true
I20260505 07:53:57.260594 1171 ts_tablet_manager.cc:1929] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.261322 1172 tablet_bootstrap.cc:654] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.254510 1423 raft_consensus.cc:493] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.262977 1176 tablet_bootstrap.cc:654] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.262805 1423 raft_consensus.cc:515] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:53:57.255751 991 leader_election.cc:343] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 772a65f4016843d18fbca25842f76de0
I20260505 07:53:57.263679 991 leader_election.cc:304] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.263936 1171 log.cc:1199] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/8b0cf443e3044e56a26b0a8a61c9cf7d
W20260505 07:53:57.264123 988 leader_election.cc:343] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 2850a301ac4f435a96742b00bf54baa9
I20260505 07:53:57.264406 1171 ts_tablet_manager.cc:1950] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:57.264390 1423 leader_election.cc:290] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:57.264814 1423 raft_consensus.cc:493] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.264838 1171 tablet_replica.cc:333] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:57.265021 1171 raft_consensus.cc:2243] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.265003 1423 raft_consensus.cc:515] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.265183 1171 raft_consensus.cc:2272] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.265727 1423 leader_election.cc:290] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.265926 1423 raft_consensus.cc:2749] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.267494 1172 tablet_bootstrap.cc:492] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:57.265971 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d4d96bdf674c2395938b9971c0ad28" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:57.268028 988 leader_election.cc:343] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:53:57.268078 1172 ts_tablet_manager.cc:1403] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.000s
I20260505 07:53:57.269611 1172 raft_consensus.cc:359] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.271812 1172 raft_consensus.cc:385] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.272073 1172 raft_consensus.cc:740] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.272416 1172 consensus_queue.cc:260] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.270360 1171 ts_tablet_manager.cc:1916] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.273391 599 master_service.cc:954] ReplaceTablet: received request to replace tablet 5f57568e1c4c4573ac18d9f621eeb4c7 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.273983 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 8b0cf443e3044e56a26b0a8a61c9cf7d (table default.Table_f4c4ec691a3445b08b83d566bd437fb8 [id=740016f529e141b58f3ea08e9b984052]) successfully deleted
I20260505 07:53:57.279399 1172 ts_tablet_manager.cc:1434] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260505 07:53:57.280161 1172 tablet_bootstrap.cc:492] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:57.282922 1176 tablet_bootstrap.cc:492] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:57.283128 1176 ts_tablet_manager.cc:1403] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.002s
W20260505 07:53:57.283113 991 leader_election.cc:343] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 2850a301ac4f435a96742b00bf54baa9
I20260505 07:53:57.283620 1172 tablet_bootstrap.cc:654] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.283653 991 leader_election.cc:304] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:53:57.284304 991 leader_election.cc:343] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 62d4d96bdf674c2395938b9971c0ad28
I20260505 07:53:57.284374 1176 raft_consensus.cc:359] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.284605 1176 raft_consensus.cc:385] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.284739 1176 raft_consensus.cc:740] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.285063 1176 consensus_queue.cc:260] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.284554 1429 raft_consensus.cc:2749] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.285725 1178 ts_tablet_manager.cc:1929] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.285804 991 leader_election.cc:304] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.285871 1178 log.cc:1199] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/222c41e6cd8b4d0093fd692618efb107
I20260505 07:53:57.286180 1429 raft_consensus.cc:2749] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.286818 1178 ts_tablet_manager.cc:1950] T 222c41e6cd8b4d0093fd692618efb107 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:57.287545 1176 ts_tablet_manager.cc:1434] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260505 07:53:57.288251 1178 tablet_replica.cc:333] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:57.290051 1176 tablet_bootstrap.cc:492] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:57.291115 1171 ts_tablet_manager.cc:1929] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.291258 1171 log.cc:1199] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/faf1140fe05e4f9493563ab8464c582b
I20260505 07:53:57.294806 1175 ts_tablet_manager.cc:1929] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.294977 1175 log.cc:1199] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/62d4d96bdf674c2395938b9971c0ad28
I20260505 07:53:57.295441 1175 ts_tablet_manager.cc:1950] T 62d4d96bdf674c2395938b9971c0ad28 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:57.296868 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 62d4d96bdf674c2395938b9971c0ad28 (table default.Table_f0316c88590245aea8815b3b08e89f45 [id=c3a480b747de444aa45b771244908352]) successfully deleted
I20260505 07:53:57.298292 595 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e]). Expected version 2 got 0
I20260505 07:53:57.298418 754 tablet_service.cc:1511] Processing CreateTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 (DEFAULT_TABLE table=default.Table_7d1747ea770445058549830f424a7aa3 [id=438fe94f1aab49919a5aae9ccab8f491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.298669 595 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]). Expected version 3 got 0
I20260505 07:53:57.298781 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7001482d90364dbdb8b9c5bfb98e1cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.299846 596 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 571257bbb7e94315b60d84d6aa4917b1 (table default.Table_087f86f68d1a411a8aba986365a3b2fe [id=6cb8bc8043474bad9f4278b7eef7c71a]). Expected version 2 got 0
I20260505 07:53:57.300580 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ab5be55234c742aea9421c70e2dad3bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.301633 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab5be55234c742aea9421c70e2dad3bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.304932 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_087f86f68d1a411a8aba986365a3b2fe" } new_table_name: "default.Table_4ab0ac01b8bf49ef96683dca41dfadd4" modify_external_catalogs: true
I20260505 07:53:57.305572 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_74517dd9ec674932b68437af722ccff7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.306620 758 tablet_service.cc:1511] Processing CreateTablet for tablet 297537a92684425d8508f7b3b4c9d379 (DEFAULT_TABLE table=default.Table_75872f5895d34e0ab0677a846b5b5926 [id=e78409bbbd334198b96cc757bc1feee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.307061 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 297537a92684425d8508f7b3b4c9d379. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:57.306753 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9044763bed1a41739ecce4def4cca1fe consensus state has no leader
W20260505 07:53:57.307811 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_74517dd9ec674932b68437af722ccff7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:57.310623 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3d83e9d4243e409794ffe8a342bce6b3 consensus state has no leader
I20260505 07:53:57.301149 891 tablet_service.cc:1511] Processing CreateTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 (DEFAULT_TABLE table=default.Table_7d1747ea770445058549830f424a7aa3 [id=438fe94f1aab49919a5aae9ccab8f491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.302363 899 tablet_service.cc:1511] Processing CreateTablet for tablet dd3fc8a9d9f04636b1f31bb88411c1da (DEFAULT_TABLE table=default.Table_1e4ca2caf1834843a4b8fc347fc080bc [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.311753 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7001482d90364dbdb8b9c5bfb98e1cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.303076 886 tablet_service.cc:1511] Processing CreateTablet for tablet 036d306d592f41cb96814aa60e30551e (DEFAULT_TABLE table=default.Table_911918ab5f5145178d2490fb80e7db21 [id=218966c8e4af4210a14adcb68269cda4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.312072 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 036d306d592f41cb96814aa60e30551e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.291671 1171 ts_tablet_manager.cc:1950] T faf1140fe05e4f9493563ab8464c582b P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:57.303507 882 tablet_service.cc:1511] Processing CreateTablet for tablet 934cea6666b1420eb12fe4510d2466fe (DEFAULT_TABLE table=default.Table_f3fff69ccbc54a58bdad7a8b5489d525 [id=c0d6550f259540e2943e2378fc4bfde5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.315997 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 934cea6666b1420eb12fe4510d2466fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.303918 890 tablet_service.cc:1511] Processing CreateTablet for tablet d622612bdc404727a27b83769bf39738 (DEFAULT_TABLE table=default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.319805 750 tablet_service.cc:1511] Processing CreateTablet for tablet de1a667b14fe494c8ee4be0f47a340be (DEFAULT_TABLE table=default.Table_085f78183e1e4000a83ccbb0ed350340 [id=037074511cd8426487cfdb5298d1ee8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.319917 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d622612bdc404727a27b83769bf39738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.320178 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de1a667b14fe494c8ee4be0f47a340be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.320336 1171 tablet_replica.cc:333] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:57.320538 1171 raft_consensus.cc:2243] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.320739 1171 raft_consensus.cc:2272] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.321238 1171 ts_tablet_manager.cc:1916] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.302013 883 tablet_service.cc:1511] Processing CreateTablet for tablet de1a667b14fe494c8ee4be0f47a340be (DEFAULT_TABLE table=default.Table_085f78183e1e4000a83ccbb0ed350340 [id=037074511cd8426487cfdb5298d1ee8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.323952 762 tablet_service.cc:1511] Processing CreateTablet for tablet dd3fc8a9d9f04636b1f31bb88411c1da (DEFAULT_TABLE table=default.Table_1e4ca2caf1834843a4b8fc347fc080bc [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.324357 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd3fc8a9d9f04636b1f31bb88411c1da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.324370 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de1a667b14fe494c8ee4be0f47a340be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.327809 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd3fc8a9d9f04636b1f31bb88411c1da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.327956 761 tablet_service.cc:1511] Processing CreateTablet for tablet 036d306d592f41cb96814aa60e30551e (DEFAULT_TABLE table=default.Table_911918ab5f5145178d2490fb80e7db21 [id=218966c8e4af4210a14adcb68269cda4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.328323 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 036d306d592f41cb96814aa60e30551e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.331537 1171 ts_tablet_manager.cc:1929] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.331672 1171 log.cc:1199] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/3272555a8c814c0ebe61ca32d715212b
I20260505 07:53:57.331892 746 tablet_service.cc:1511] Processing CreateTablet for tablet 934cea6666b1420eb12fe4510d2466fe (DEFAULT_TABLE table=default.Table_f3fff69ccbc54a58bdad7a8b5489d525 [id=c0d6550f259540e2943e2378fc4bfde5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.332360 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 934cea6666b1420eb12fe4510d2466fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.335906 749 tablet_service.cc:1511] Processing CreateTablet for tablet d622612bdc404727a27b83769bf39738 (DEFAULT_TABLE table=default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.336323 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d622612bdc404727a27b83769bf39738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.337692 1171 ts_tablet_manager.cc:1950] T 3272555a8c814c0ebe61ca32d715212b P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:57.339733 744 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ec347e754ab4b7282e68b15a9bbf993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.339967 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.349272 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet faf1140fe05e4f9493563ab8464c582b (table default.Table_da8a2a5c54bf40a784a8a01b7245b280 [id=2170bb22bfb747c5b4f24ba83810f89e]) successfully deleted
I20260505 07:53:57.350159 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 3272555a8c814c0ebe61ca32d715212b (table default.Table_70e3149cee264c889c7677c3757ef266 [id=27e306dc7bda4ca0b28f72e8636b91d5]) successfully deleted
W20260505 07:53:57.350608 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 already in progress: creating tablet
W20260505 07:53:57.351038 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 4258df74ea0841449846cdf1f6e00b9b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4258df74ea0841449846cdf1f6e00b9b already in progress: creating tablet
I20260505 07:53:57.362102 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8cdf9d3d3e6349f4b8d3536f78451100" } new_table_name: "default.Table_829cc346ac834637ab2e2198944a8550" modify_external_catalogs: true
I20260505 07:53:57.362617 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_e9ba8a72d90d4a3da2c06b684fe35033"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.363097 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e9ba8a72d90d4a3da2c06b684fe35033 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.364611 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_cc284917d2aa41d9956affce2c946825"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.365108 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc284917d2aa41d9956affce2c946825 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.366245 883 tablet_service.cc:1558] Processing DeleteTablet for tablet c6dcbebddcdc4ad9a788c01134e1101c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.366609 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 6429919aba17447dbdb393176360bbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.369190 1175 tablet_replica.cc:333] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:57.369539 1175 raft_consensus.cc:2243] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.369782 1175 raft_consensus.cc:2272] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.369313 1176 tablet_bootstrap.cc:654] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.294320 1429 raft_consensus.cc:493] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.370642 1429 raft_consensus.cc:515] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.371430 1175 ts_tablet_manager.cc:1916] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.371484 1429 leader_election.cc:290] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.325484 1423 raft_consensus.cc:493] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.371685 1178 raft_consensus.cc:2243] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.371735 1423 raft_consensus.cc:515] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.371881 1178 raft_consensus.cc:2272] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.372265 1423 leader_election.cc:290] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.372426 1178 ts_tablet_manager.cc:1916] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.373147 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cfe212cb6c14ce3a63b6ee4275f66f1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:57.373333 782 raft_consensus.cc:2468] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:53:57.373703 991 leader_election.cc:343] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8cfe212cb6c14ce3a63b6ee4275f66f1
W20260505 07:53:57.374446 991 leader_election.cc:343] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 5b124eda66364b7a81426934b98bca48
I20260505 07:53:57.374676 988 leader_election.cc:304] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
W20260505 07:53:57.375083 988 leader_election.cc:343] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 5b124eda66364b7a81426934b98bca48
I20260505 07:53:57.375193 988 leader_election.cc:304] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.375762 1423 raft_consensus.cc:2804] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:57.375854 1423 raft_consensus.cc:493] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:57.375949 1423 raft_consensus.cc:3060] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:57.378842 1423 raft_consensus.cc:515] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.380069 1429 raft_consensus.cc:2749] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:53:57.380481 991 leader_election.cc:343] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8cfe212cb6c14ce3a63b6ee4275f66f1
I20260505 07:53:57.380990 1423 leader_election.cc:290] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.381222 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8cfe212cb6c14ce3a63b6ee4275f66f1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:53:57.381467 781 raft_consensus.cc:3060] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:57.385556 1364 raft_consensus.cc:493] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.385700 1364 raft_consensus.cc:515] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.386286 1364 leader_election.cc:290] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.301626 897 tablet_service.cc:1511] Processing CreateTablet for tablet 297537a92684425d8508f7b3b4c9d379 (DEFAULT_TABLE table=default.Table_75872f5895d34e0ab0677a846b5b5926 [id=e78409bbbd334198b96cc757bc1feee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.386844 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 297537a92684425d8508f7b3b4c9d379. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:57.386948 991 leader_election.cc:343] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: fea61fad19b34ffaa0b9df3a51979b18
W20260505 07:53:57.387619 988 leader_election.cc:343] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: fea61fad19b34ffaa0b9df3a51979b18
I20260505 07:53:57.387729 988 leader_election.cc:304] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.387760 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 202a237ca580482c98803683a77d96d2
I20260505 07:53:57.387955 781 raft_consensus.cc:2468] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:53:57.388348 988 leader_election.cc:304] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:53:57.388702 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7eb9890f5e7349748b35f1134174580e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.389283 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7eb9890f5e7349748b35f1134174580e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.389345 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 202a237ca580482c98803683a77d96d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.390749 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f3a7ead203124b49a437c53941ec4d58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.391163 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3a7ead203124b49a437c53941ec4d58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.392015 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 23fa401bb4634022bca3c6b23421e4d5
I20260505 07:53:57.392597 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 872362aecf6a44b59e586b1fa7637c8b from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.391968 1176 tablet_bootstrap.cc:492] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:57.392757 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 5f57568e1c4c4573ac18d9f621eeb4c7
I20260505 07:53:57.392910 1176 ts_tablet_manager.cc:1403] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.104s user 0.006s sys 0.000s
I20260505 07:53:57.393105 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_09814f6166bb4e6e9978387328078099"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.393214 599 catalog_manager.cc:6741] ReplaceTablet: tablet 5f57568e1c4c4573ac18d9f621eeb4c7 deleted and replaced by tablet 727cacd242844455915bd9963d2db19f
I20260505 07:53:57.393457 599 master_service.cc:954] ReplaceTablet: received request to replace tablet bde59d52fd234343b1ca13a7d4aae6bf from {username='slave'} at 127.0.0.1:47238
W20260505 07:53:57.393522 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_09814f6166bb4e6e9978387328078099 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.394058 1176 raft_consensus.cc:359] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.394253 1176 raft_consensus.cc:385] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.395833 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f57568e1c4c4573ac18d9f621eeb4c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 727cacd242844455915bd9963d2db19f) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.395685 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.394342 1176 raft_consensus.cc:740] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.396339 882 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f57568e1c4c4573ac18d9f621eeb4c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 727cacd242844455915bd9963d2db19f) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.396548 1176 consensus_queue.cc:260] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.399411 1176 ts_tablet_manager.cc:1434] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:53:57.399833 883 tablet_service.cc:1558] Processing DeleteTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.400068 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 202a237ca580482c98803683a77d96d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.400527 1176 tablet_bootstrap.cc:492] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:57.400744 1425 raft_consensus.cc:2749] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.400967 1425 raft_consensus.cc:2804] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:57.401134 1425 raft_consensus.cc:697] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:53:57.401448 1425 consensus_queue.cc:237] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.310806 1172 tablet_bootstrap.cc:492] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:57.402611 1172 ts_tablet_manager.cc:1403] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.123s user 0.004s sys 0.000s
I20260505 07:53:57.403366 1172 raft_consensus.cc:359] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.402637 1175 ts_tablet_manager.cc:1929] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.403620 1172 raft_consensus.cc:385] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.403703 1172 raft_consensus.cc:740] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.403959 1172 consensus_queue.cc:260] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.404341 1175 log.cc:1199] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/2ec347e754ab4b7282e68b15a9bbf993
I20260505 07:53:57.405062 1175 ts_tablet_manager.cc:1950] T 2ec347e754ab4b7282e68b15a9bbf993 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:57.405622 1172 ts_tablet_manager.cc:1434] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260505 07:53:57.406651 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 (DEFAULT_TABLE table=default.Table_7d1747ea770445058549830f424a7aa3 [id=438fe94f1aab49919a5aae9ccab8f491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.407490 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 2ec347e754ab4b7282e68b15a9bbf993 (table default.Table_4d8c9530e5904e54ac3bdb2a82083180 [id=7e62bbf111c94a01b324b800bd6ced0b]) successfully deleted
W20260505 07:53:57.407894 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 571257bbb7e94315b60d84d6aa4917b1 consensus state has no leader
I20260505 07:53:57.408205 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_499d356daab040e29ca2f5eb2941d790" } modify_external_catalogs: true
I20260505 07:53:57.408407 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_499d356daab040e29ca2f5eb2941d790" } modify_external_catalogs: true
I20260505 07:53:57.406313 1176 tablet_bootstrap.cc:654] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.408849 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 62d4bc3154704ba2bfde83bca10f812d
I20260505 07:53:57.409122 1175 tablet_replica.cc:333] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:57.409883 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4bc3154704ba2bfde83bca10f812d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.410259 1175 raft_consensus.cc:2243] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.410653 1175 raft_consensus.cc:2272] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.410876 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ba3de8258d7e46c1ad3a509b8e00b806"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.411628 1175 ts_tablet_manager.cc:1916] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:57.411702 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba3de8258d7e46c1ad3a509b8e00b806 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:57.412353 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 42360f8ee4ff47c2af8d287f792207de consensus state has no leader
I20260505 07:53:57.412765 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1e4ca2caf1834843a4b8fc347fc080bc" } new_table_name: "default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f" modify_external_catalogs: true
I20260505 07:53:57.414690 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_422080d5c75f422fb88c26b76ffc61c4" } modify_external_catalogs: true
I20260505 07:53:57.414870 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_422080d5c75f422fb88c26b76ffc61c4" } modify_external_catalogs: true
I20260505 07:53:57.415676 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4bc3154704ba2bfde83bca10f812d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.417292 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet bde59d52fd234343b1ca13a7d4aae6bf
I20260505 07:53:57.417739 599 catalog_manager.cc:6741] ReplaceTablet: tablet bde59d52fd234343b1ca13a7d4aae6bf deleted and replaced by tablet 3a73dbb979344805876efbafbf1b2985
I20260505 07:53:57.418175 758 tablet_service.cc:1558] Processing DeleteTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf with delete_type TABLET_DATA_DELETED (replaced by tablet 3a73dbb979344805876efbafbf1b2985) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.418812 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 297537a92684425d8508f7b3b4c9d379 (DEFAULT_TABLE table=default.Table_75872f5895d34e0ab0677a846b5b5926 [id=e78409bbbd334198b96cc757bc1feee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.418926 899 tablet_service.cc:1558] Processing DeleteTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf with delete_type TABLET_DATA_DELETED (replaced by tablet 3a73dbb979344805876efbafbf1b2985) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.419749 1032 tablet_service.cc:1511] Processing CreateTablet for tablet de1a667b14fe494c8ee4be0f47a340be (DEFAULT_TABLE table=default.Table_085f78183e1e4000a83ccbb0ed350340 [id=037074511cd8426487cfdb5298d1ee8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.420650 1024 tablet_service.cc:1511] Processing CreateTablet for tablet dd3fc8a9d9f04636b1f31bb88411c1da (DEFAULT_TABLE table=default.Table_1e4ca2caf1834843a4b8fc347fc080bc [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.421590 1029 tablet_service.cc:1511] Processing CreateTablet for tablet 036d306d592f41cb96814aa60e30551e (DEFAULT_TABLE table=default.Table_911918ab5f5145178d2490fb80e7db21 [id=218966c8e4af4210a14adcb68269cda4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.423537 1176 tablet_bootstrap.cc:492] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:57.423784 1176 ts_tablet_manager.cc:1403] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.005s
I20260505 07:53:57.424160 1020 tablet_service.cc:1511] Processing CreateTablet for tablet 934cea6666b1420eb12fe4510d2466fe (DEFAULT_TABLE table=default.Table_f3fff69ccbc54a58bdad7a8b5489d525 [id=c0d6550f259540e2943e2378fc4bfde5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.424830 1176 raft_consensus.cc:359] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.425125 1176 raft_consensus.cc:385] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.425248 1176 raft_consensus.cc:740] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.425562 1176 consensus_queue.cc:260] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.426455 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 222c41e6cd8b4d0093fd692618efb107 (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]) successfully deleted
I20260505 07:53:57.428061 1176 ts_tablet_manager.cc:1434] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
W20260505 07:53:57.428619 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet dd3fc8a9d9f04636b1f31bb88411c1da consensus state has no leader
I20260505 07:53:57.429427 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet e1a9420a81844d2092381123d9297661
I20260505 07:53:57.429768 1176 tablet_bootstrap.cc:492] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:57.430614 758 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a9420a81844d2092381123d9297661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.431754 899 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a9420a81844d2092381123d9297661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.433620 1175 ts_tablet_manager.cc:1929] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.435478 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_e7ad0f00a4fc456e944f70e21d505fd5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.435650 1025 tablet_service.cc:1511] Processing CreateTablet for tablet d622612bdc404727a27b83769bf39738 (DEFAULT_TABLE table=default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.435990 1175 log.cc:1199] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/8cfe212cb6c14ce3a63b6ee4275f66f1
W20260505 07:53:57.436017 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7ad0f00a4fc456e944f70e21d505fd5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.436460 1175 ts_tablet_manager.cc:1950] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:57.435853 1176 tablet_bootstrap.cc:654] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.439105 1175 tablet_replica.cc:333] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:57.438853 1424 raft_consensus.cc:493] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.439368 1175 raft_consensus.cc:2243] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.439530 1175 raft_consensus.cc:2272] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.440375 1175 ts_tablet_manager.cc:1916] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.450064 1176 tablet_bootstrap.cc:492] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:57.450600 1176 ts_tablet_manager.cc:1403] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.002s
I20260505 07:53:57.451228 1172 tablet_bootstrap.cc:492] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:57.451877 1176 raft_consensus.cc:359] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.452186 1176 raft_consensus.cc:385] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.452575 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 (table default.Table_2b834837c1d54b429b8d9a937f56d726 [id=49a5c863608b402a9ca94c01e5a5c09f]) successfully deleted
W20260505 07:53:57.453804 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 202a237ca580482c98803683a77d96d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 202a237ca580482c98803683a77d96d2 already in progress: creating tablet
I20260505 07:53:57.454700 594 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]). Expected version 3 got 0
I20260505 07:53:57.454962 758 tablet_service.cc:1511] Processing CreateTablet for tablet 46274a44a06b4306b5dac43c66c85a80 (DEFAULT_TABLE table=default.Table_614ea89c84d342f0a1c4912e8e77fe47 [id=f1230f25280e45269ffbd95bbe8dbbfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.455374 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46274a44a06b4306b5dac43c66c85a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.455351 899 tablet_service.cc:1511] Processing CreateTablet for tablet 46274a44a06b4306b5dac43c66c85a80 (DEFAULT_TABLE table=default.Table_614ea89c84d342f0a1c4912e8e77fe47 [id=f1230f25280e45269ffbd95bbe8dbbfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.455907 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46274a44a06b4306b5dac43c66c85a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.461462 746 tablet_service.cc:1511] Processing CreateTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 (DEFAULT_TABLE table=default.Table_0b3a73b640a644579fa864cce1a7ecbe [id=6c1e059a94014956ab0b1d6c750d8494]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.461920 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36b33d265ba04e93993c7e0a4efea1a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.463907 762 tablet_service.cc:1511] Processing CreateTablet for tablet 21480c42b73e4bf0ab2453b085a867c0 (DEFAULT_TABLE table=default.Table_db54454473d84a83b6c77d06a1f09618 [id=0faa6676bab04919984b580299143271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.464321 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21480c42b73e4bf0ab2453b085a867c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.464270 897 tablet_service.cc:1511] Processing CreateTablet for tablet 21480c42b73e4bf0ab2453b085a867c0 (DEFAULT_TABLE table=default.Table_db54454473d84a83b6c77d06a1f09618 [id=0faa6676bab04919984b580299143271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.464526 890 tablet_service.cc:1511] Processing CreateTablet for tablet 26f3544e74d941d286a2a0317b941ab6 (DEFAULT_TABLE table=default.Table_82c755fca0b64337a44a63876e04fa68 [id=ded6ca9533da4aa197bbd8c9b4a72066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.464655 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21480c42b73e4bf0ab2453b085a867c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.464874 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26f3544e74d941d286a2a0317b941ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.465159 882 tablet_service.cc:1511] Processing CreateTablet for tablet e29ba55a846548679cb3b86305d0d6ed (DEFAULT_TABLE table=default.Table_fd75ee1f117344169ccafd58dc18408f [id=5cd6a3d7d65d4a43943fa891ca480d7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.465513 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e29ba55a846548679cb3b86305d0d6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.467972 749 tablet_service.cc:1511] Processing CreateTablet for tablet e29ba55a846548679cb3b86305d0d6ed (DEFAULT_TABLE table=default.Table_fd75ee1f117344169ccafd58dc18408f [id=5cd6a3d7d65d4a43943fa891ca480d7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.468338 883 tablet_service.cc:1511] Processing CreateTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 (DEFAULT_TABLE table=default.Table_0b3a73b640a644579fa864cce1a7ecbe [id=6c1e059a94014956ab0b1d6c750d8494]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.468405 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e29ba55a846548679cb3b86305d0d6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.468466 886 tablet_service.cc:1511] Processing CreateTablet for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (DEFAULT_TABLE table=default.Table_fd8baf98a0f64d7fbd3d61d32af60991 [id=3f62e5dc85a04f1295d1d2b84fff8f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.468700 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36b33d265ba04e93993c7e0a4efea1a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.468907 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4e5af08ddc4c77b5640b8295b236b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.471691 761 tablet_service.cc:1511] Processing CreateTablet for tablet 26f3544e74d941d286a2a0317b941ab6 (DEFAULT_TABLE table=default.Table_82c755fca0b64337a44a63876e04fa68 [id=ded6ca9533da4aa197bbd8c9b4a72066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.472061 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26f3544e74d941d286a2a0317b941ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.475912 754 tablet_service.cc:1511] Processing CreateTablet for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (DEFAULT_TABLE table=default.Table_fd8baf98a0f64d7fbd3d61d32af60991 [id=3f62e5dc85a04f1295d1d2b84fff8f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.476301 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4e5af08ddc4c77b5640b8295b236b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.479691 1171 tablet_replica.cc:333] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:57.479933 1171 raft_consensus.cc:2243] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.480100 1171 raft_consensus.cc:2272] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.480886 1171 ts_tablet_manager.cc:1916] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.490527 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 323269e708964848a94c64e6a781a860 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.501057 594 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]). Expected version 3 got 0
I20260505 07:53:57.508174 883 tablet_service.cc:1558] Processing DeleteTablet for tablet 32c008bd99224a7595cecc607e5496a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a9494c3b6a99487596a030856b371b68) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.513295 1176 raft_consensus.cc:740] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.513727 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 8b57c1c618484d5d86a846fb982a2240 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.513984 1171 ts_tablet_manager.cc:1929] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.514128 1171 log.cc:1199] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/c6dcbebddcdc4ad9a788c01134e1101c
I20260505 07:53:57.514528 1171 ts_tablet_manager.cc:1950] T c6dcbebddcdc4ad9a788c01134e1101c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:57.514034 1176 consensus_queue.cc:260] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.515513 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet c6dcbebddcdc4ad9a788c01134e1101c (table default.Table_ba04ccad67434c7884329b23efea4bfd [id=36cc8dc1683b4ef194ec299e4fed4de9]) successfully deleted
I20260505 07:53:57.519132 1176 ts_tablet_manager.cc:1434] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.068s user 0.002s sys 0.000s
I20260505 07:53:57.521947 1176 tablet_bootstrap.cc:492] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:57.520915 1171 tablet_replica.cc:333] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:57.522799 1171 raft_consensus.cc:2243] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.522977 1171 raft_consensus.cc:2272] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.523720 1171 ts_tablet_manager.cc:1916] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.526054 1176 tablet_bootstrap.cc:654] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.526115 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 956b5e05bda648349ba26d5c1ea3b279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.527338 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet ce9d01b0b5464a65966f26839b48aaf1 with delete_type TABLET_DATA_DELETED (replaced by tablet 17327cee69a24e0cb493ff55f948bf6c) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.528545 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b124eda66364b7a81426934b98bca48 with delete_type TABLET_DATA_DELETED (replaced by tablet 3781d0bbcd1b4189bd836eeae4236f40) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.529557 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.530424 1171 ts_tablet_manager.cc:1929] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.530546 1171 log.cc:1199] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/6429919aba17447dbdb393176360bbb0
I20260505 07:53:57.530582 1175 ts_tablet_manager.cc:1929] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.530664 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 1765a95cc3414c73b7e3918913a35937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.530757 1175 log.cc:1199] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/5f57568e1c4c4573ac18d9f621eeb4c7
I20260505 07:53:57.530925 1171 ts_tablet_manager.cc:1950] T 6429919aba17447dbdb393176360bbb0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:57.531208 1175 ts_tablet_manager.cc:1950] T 5f57568e1c4c4573ac18d9f621eeb4c7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:57.531924 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d6490f568024510ab843ec5205a4d1a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.531945 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 6429919aba17447dbdb393176360bbb0 (table default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]) successfully deleted
I20260505 07:53:57.532240 1171 tablet_replica.cc:333] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:57.532449 1171 raft_consensus.cc:2243] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.532612 1171 raft_consensus.cc:2272] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.532811 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 5f57568e1c4c4573ac18d9f621eeb4c7 (table default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]) successfully deleted
I20260505 07:53:57.533563 1171 ts_tablet_manager.cc:1916] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:57.533605 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 23fa401bb4634022bca3c6b23421e4d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23fa401bb4634022bca3c6b23421e4d5 already in progress: creating tablet
W20260505 07:53:57.534389 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 62d4bc3154704ba2bfde83bca10f812d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62d4bc3154704ba2bfde83bca10f812d already in progress: creating tablet
W20260505 07:53:57.534865 1437 log.cc:927] Time spent T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: Append to log took a long time: real 0.092s user 0.000s sys 0.000s
W20260505 07:53:57.535123 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet bde59d52fd234343b1ca13a7d4aae6bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bde59d52fd234343b1ca13a7d4aae6bf already in progress: creating tablet
W20260505 07:53:57.535763 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e1a9420a81844d2092381123d9297661 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e1a9420a81844d2092381123d9297661 already in progress: creating tablet
I20260505 07:53:57.535972 1181 tablet_bootstrap.cc:492] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.536110 1181 ts_tablet_manager.cc:1403] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.320s user 0.029s sys 0.045s
I20260505 07:53:57.536855 1181 raft_consensus.cc:359] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.537048 1181 raft_consensus.cc:385] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.537134 1364 raft_consensus.cc:493] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.537155 1181 raft_consensus.cc:740] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.537278 1364 raft_consensus.cc:515] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.537372 1181 consensus_queue.cc:260] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.537843 1181 ts_tablet_manager.cc:1434] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:53:57.537899 1364 leader_election.cc:290] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.539832 1181 tablet_bootstrap.cc:492] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.540233 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de1a667b14fe494c8ee4be0f47a340be. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:57.540333 988 leader_election.cc:343] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: cf95f4331e4540ae84e329cf577427b8
W20260505 07:53:57.540613 991 leader_election.cc:343] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: cf95f4331e4540ae84e329cf577427b8
I20260505 07:53:57.540753 991 leader_election.cc:304] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.446481 1424 raft_consensus.cc:515] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.541389 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 934cea6666b1420eb12fe4510d2466fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.543085 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d533362919954c83aa119d8607d882d6" } new_table_name: "default.Table_633da5b39da34f5c88163c9936fd0212" modify_external_catalogs: true
I20260505 07:53:57.543176 593 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7f72af6be89b4942a09acb0dc9615952" } new_table_name: "default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c" modify_external_catalogs: true
I20260505 07:53:57.546080 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d4d6e315fb8340e9b629d876d050ef05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.546764 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4d6e315fb8340e9b629d876d050ef05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.547919 1055 raft_consensus.cc:3060] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:57.548058 1171 ts_tablet_manager.cc:1929] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.548276 1171 log.cc:1199] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/5f57568e1c4c4573ac18d9f621eeb4c7
I20260505 07:53:57.548646 1171 ts_tablet_manager.cc:1950] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:57.549834 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 5f57568e1c4c4573ac18d9f621eeb4c7 (table default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]) successfully deleted
W20260505 07:53:57.550475 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 23fa401bb4634022bca3c6b23421e4d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23fa401bb4634022bca3c6b23421e4d5 already in progress: creating tablet
W20260505 07:53:57.551007 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 202a237ca580482c98803683a77d96d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 202a237ca580482c98803683a77d96d2 already in progress: creating tablet
W20260505 07:53:57.551554 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 62d4bc3154704ba2bfde83bca10f812d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62d4bc3154704ba2bfde83bca10f812d already in progress: creating tablet
W20260505 07:53:57.552214 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet bde59d52fd234343b1ca13a7d4aae6bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bde59d52fd234343b1ca13a7d4aae6bf already in progress: creating tablet
I20260505 07:53:57.552511 1171 tablet_replica.cc:333] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:57.552696 1171 raft_consensus.cc:2243] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.552888 1171 raft_consensus.cc:2272] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.553910 1171 ts_tablet_manager.cc:1916] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.554625 1364 raft_consensus.cc:2749] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.557652 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d622612bdc404727a27b83769bf39738. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.558035 1181 tablet_bootstrap.cc:654] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.559415 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd3fc8a9d9f04636b1f31bb88411c1da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.559800 1171 ts_tablet_manager.cc:1929] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.559935 1171 log.cc:1199] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e1a9420a81844d2092381123d9297661
I20260505 07:53:57.560281 1171 ts_tablet_manager.cc:1950] T e1a9420a81844d2092381123d9297661 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:57.561448 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e1a9420a81844d2092381123d9297661 (table default.Table_422080d5c75f422fb88c26b76ffc61c4 [id=0d8de70d77c54c2491b257d09c8474c8]) successfully deleted
I20260505 07:53:57.561830 1171 tablet_replica.cc:333] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:57.562078 1171 raft_consensus.cc:2243] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.562235 1171 raft_consensus.cc:2272] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.563001 1171 ts_tablet_manager.cc:1916] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.565871 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7001482d90364dbdb8b9c5bfb98e1cd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.567762 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca4cf248adb4f768c5586d72614ad42" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:53:57.568997 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ca4cf248adb4f768c5586d72614ad42" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:57.569458 1171 ts_tablet_manager.cc:1929] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.569599 1171 log.cc:1199] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/32c008bd99224a7595cecc607e5496a9
I20260505 07:53:57.569932 1171 ts_tablet_manager.cc:1950] T 32c008bd99224a7595cecc607e5496a9 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:57.570065 988 leader_election.cc:304] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.570964 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 32c008bd99224a7595cecc607e5496a9 (table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]) successfully deleted
I20260505 07:53:57.567307 1424 leader_election.cc:290] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.573810 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 297537a92684425d8508f7b3b4c9d379. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.581346 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 036d306d592f41cb96814aa60e30551e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.584605 598 catalog_manager.cc:5671] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:53:57.585036 598 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]). Expected version 3 got 0
I20260505 07:53:57.585340 920 raft_consensus.cc:1275] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Refusing update from remote peer eac7e1e8caba43cdb4df80cbb80e891c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260505 07:53:57.585590 1052 raft_consensus.cc:3060] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:57.586690 1379 consensus_queue.cc:1048] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:53:57.588424 1052 raft_consensus.cc:1275] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Refusing update from remote peer eac7e1e8caba43cdb4df80cbb80e891c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260505 07:53:57.591100 599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1496aa36a9c0478fa72f40d4595c2398" } modify_external_catalogs: true
I20260505 07:53:57.591284 599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1496aa36a9c0478fa72f40d4595c2398" } modify_external_catalogs: true
I20260505 07:53:57.592700 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_5a41e60184ab44a99342444292f0cc5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.593307 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5a41e60184ab44a99342444292f0cc5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.596735 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ac5741221f7e4f54bffba1c88d5cf5c1" } modify_external_catalogs: true
I20260505 07:53:57.596982 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ac5741221f7e4f54bffba1c88d5cf5c1" } modify_external_catalogs: true
I20260505 07:53:57.598508 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 56cbbb89f6b84407b2c344ec62402b03 with delete_type TABLET_DATA_DELETED (replaced by tablet 9044763bed1a41739ecce4def4cca1fe) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.598842 1175 tablet_replica.cc:333] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:57.599053 1175 raft_consensus.cc:2243] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.599246 1175 raft_consensus.cc:2272] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.600248 1175 ts_tablet_manager.cc:1916] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.604477 1379 consensus_queue.cc:1048] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:53:57.609028 1175 ts_tablet_manager.cc:1929] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.609198 1175 log.cc:1199] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/56cbbb89f6b84407b2c344ec62402b03
I20260505 07:53:57.609766 1175 ts_tablet_manager.cc:1950] T 56cbbb89f6b84407b2c344ec62402b03 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:57.609913 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_042d1a9abf8948aebac31e5f4f810019" } new_table_name: "default.Table_fa6246cc3ccd4d7082becde923f88828" modify_external_catalogs: true
I20260505 07:53:57.609994 1181 tablet_bootstrap.cc:492] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.610175 1181 ts_tablet_manager.cc:1403] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.070s user 0.003s sys 0.007s
I20260505 07:53:57.611600 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 56cbbb89f6b84407b2c344ec62402b03 (table default.Table_c41ba7470a784645bdddd3a262d4bb4e [id=bd575286f4e94df3b335b5e79d8f971c]) successfully deleted
I20260505 07:53:57.611954 1172 tablet_bootstrap.cc:654] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.617144 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 323269e708964848a94c64e6a781a860
I20260505 07:53:57.617627 596 catalog_manager.cc:6741] ReplaceTablet: tablet 323269e708964848a94c64e6a781a860 deleted and replaced by tablet e010f23c4df647ab863e03c3cfed7898
I20260505 07:53:57.618322 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 323269e708964848a94c64e6a781a860 with delete_type TABLET_DATA_DELETED (replaced by tablet e010f23c4df647ab863e03c3cfed7898) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.618438 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 397d33e908af4a0bb8dde26d273be01a
W20260505 07:53:57.620066 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 323269e708964848a94c64e6a781a860 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 323269e708964848a94c64e6a781a860 already in progress: creating tablet
I20260505 07:53:57.620563 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 397d33e908af4a0bb8dde26d273be01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:57.621256 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 397d33e908af4a0bb8dde26d273be01a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 397d33e908af4a0bb8dde26d273be01a already in progress: creating tablet
I20260505 07:53:57.622807 883 tablet_service.cc:1558] Processing DeleteTablet for tablet 397d33e908af4a0bb8dde26d273be01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:57.624748 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 397d33e908af4a0bb8dde26d273be01a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 397d33e908af4a0bb8dde26d273be01a already in progress: creating tablet
I20260505 07:53:57.640075 1181 raft_consensus.cc:359] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.640545 1181 raft_consensus.cc:385] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.640677 1181 raft_consensus.cc:740] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.641469 1181 consensus_queue.cc:260] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.642176 1181 ts_tablet_manager.cc:1434] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.032s user 0.001s sys 0.001s
I20260505 07:53:57.646171 1181 tablet_bootstrap.cc:492] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.647717 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet cf95f4331e4540ae84e329cf577427b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:53 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.647750 1459 tablet.cc:1724] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:53:57.648228 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 202a237ca580482c98803683a77d96d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.648666 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.649083 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 5f57568e1c4c4573ac18d9f621eeb4c7 with delete_type TABLET_DATA_DELETED (replaced by tablet 727cacd242844455915bd9963d2db19f) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.655345 1237 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.61:33565
I20260505 07:53:57.659480 1237 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-1/data/info.pb
I20260505 07:53:57.662036 1465 tablet.cc:1724] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:53:57.662760 1172 tablet_bootstrap.cc:492] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:57.665187 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu as pid 1237
I20260505 07:53:57.666122 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4bc3154704ba2bfde83bca10f812d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.666471 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf with delete_type TABLET_DATA_DELETED (replaced by tablet 3a73dbb979344805876efbafbf1b2985) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.666791 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a9420a81844d2092381123d9297661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.667375 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 46274a44a06b4306b5dac43c66c85a80 (DEFAULT_TABLE table=default.Table_614ea89c84d342f0a1c4912e8e77fe47 [id=f1230f25280e45269ffbd95bbe8dbbfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.667814 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 46274a44a06b4306b5dac43c66c85a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.674827 1172 ts_tablet_manager.cc:1403] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.224s user 0.002s sys 0.004s
I20260505 07:53:57.679538 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 26f3544e74d941d286a2a0317b941ab6 (DEFAULT_TABLE table=default.Table_82c755fca0b64337a44a63876e04fa68 [id=ded6ca9533da4aa197bbd8c9b4a72066]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.679970 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26f3544e74d941d286a2a0317b941ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.680008 1172 raft_consensus.cc:359] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.681176 1172 raft_consensus.cc:385] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.681818 1172 raft_consensus.cc:740] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.683302 1172 consensus_queue.cc:260] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.690908 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 21480c42b73e4bf0ab2453b085a867c0 (DEFAULT_TABLE table=default.Table_db54454473d84a83b6c77d06a1f09618 [id=0faa6676bab04919984b580299143271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.692433 1425 raft_consensus.cc:493] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.692600 1425 raft_consensus.cc:515] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.693758 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dab02a546b944f799045723a515118f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:57.693863 1425 leader_election.cc:290] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:53:57.694564 991 leader_election.cc:343] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 5dab02a546b944f799045723a515118f
W20260505 07:53:57.695010 988 leader_election.cc:343] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.696322 988 leader_election.cc:304] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.696746 1425 raft_consensus.cc:2749] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.697664 1020 tablet_service.cc:1511] Processing CreateTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 (DEFAULT_TABLE table=default.Table_0b3a73b640a644579fa864cce1a7ecbe [id=6c1e059a94014956ab0b1d6c750d8494]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.698002 1172 ts_tablet_manager.cc:1434] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20260505 07:53:57.699028 1466 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.61:33565 every 8 connection(s)
I20260505 07:53:57.705691 1172 tablet_bootstrap.cc:492] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:57.709566 1467 sys_catalog.cc:263] Verifying existing consensus state
I20260505 07:53:57.708704 886 tablet_service.cc:1558] Processing DeleteTablet for tablet 323269e708964848a94c64e6a781a860 with delete_type TABLET_DATA_DELETED (replaced by tablet e010f23c4df647ab863e03c3cfed7898) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:57.712590 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 323269e708964848a94c64e6a781a860 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 323269e708964848a94c64e6a781a860 already in progress: creating tablet
I20260505 07:53:57.713267 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_614ea89c84d342f0a1c4912e8e77fe47" } modify_external_catalogs: true
I20260505 07:53:57.713451 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_614ea89c84d342f0a1c4912e8e77fe47" } modify_external_catalogs: true
I20260505 07:53:57.714133 1467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Bootstrap starting.
I20260505 07:53:57.715036 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_2717154d3561495692818d7755d53c41"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.715664 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2717154d3561495692818d7755d53c41 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.720748 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21480c42b73e4bf0ab2453b085a867c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.726662 1467 log.cc:826] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Log is configured to *not* fsync() on all Append() calls
W20260505 07:53:57.728044 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e278b86e7b5d444ebad8d0f6921d4c62 consensus state has no leader
W20260505 07:53:57.729584 593 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf consensus state has no leader
I20260505 07:53:57.731504 1181 tablet_bootstrap.cc:654] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.732051 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 36b33d265ba04e93993c7e0a4efea1a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.732937 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (DEFAULT_TABLE table=default.Table_fd8baf98a0f64d7fbd3d61d32af60991 [id=3f62e5dc85a04f1295d1d2b84fff8f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.733803 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet f92c53c68a4b4e319e98b84128e8e6a4
I20260505 07:53:57.734925 762 tablet_service.cc:1558] Processing DeleteTablet for tablet f92c53c68a4b4e319e98b84128e8e6a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.735107 1176 tablet_bootstrap.cc:492] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:57.735257 1176 ts_tablet_manager.cc:1403] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.214s user 0.026s sys 0.037s
W20260505 07:53:57.736024 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet f92c53c68a4b4e319e98b84128e8e6a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f92c53c68a4b4e319e98b84128e8e6a4 already in progress: creating tablet
I20260505 07:53:57.736088 1176 raft_consensus.cc:359] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.736270 1176 raft_consensus.cc:385] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.736346 1176 raft_consensus.cc:740] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.736420 886 tablet_service.cc:1558] Processing DeleteTablet for tablet f92c53c68a4b4e319e98b84128e8e6a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.736698 1176 consensus_queue.cc:260] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.737293 1176 ts_tablet_manager.cc:1434] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:53:57.737635 1176 tablet_bootstrap.cc:492] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:57.738121 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ec347e754ab4b7282e68b15a9bbf993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.738700 1178 ts_tablet_manager.cc:1929] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.738880 1178 log.cc:1199] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/772a65f4016843d18fbca25842f76de0
I20260505 07:53:57.739213 1176 tablet_bootstrap.cc:654] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.735730 1025 tablet_service.cc:1511] Processing CreateTablet for tablet e29ba55a846548679cb3b86305d0d6ed (DEFAULT_TABLE table=default.Table_fd75ee1f117344169ccafd58dc18408f [id=5cd6a3d7d65d4a43943fa891ca480d7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.741704 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e29ba55a846548679cb3b86305d0d6ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.741919 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet d622612bdc404727a27b83769bf39738
W20260505 07:53:57.743271 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet f92c53c68a4b4e319e98b84128e8e6a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f92c53c68a4b4e319e98b84128e8e6a4 already in progress: creating tablet
I20260505 07:53:57.743463 762 tablet_service.cc:1558] Processing DeleteTablet for tablet d622612bdc404727a27b83769bf39738 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.743499 897 tablet_service.cc:1558] Processing DeleteTablet for tablet d622612bdc404727a27b83769bf39738 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:57.744094 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet d622612bdc404727a27b83769bf39738 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d622612bdc404727a27b83769bf39738 already in progress: creating tablet
W20260505 07:53:57.744369 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet d622612bdc404727a27b83769bf39738 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d622612bdc404727a27b83769bf39738 already in progress: creating tablet
I20260505 07:53:57.744760 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet f92c53c68a4b4e319e98b84128e8e6a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.744681 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_8ef2c24cce2148ecb1409ecaa29a8130"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.745541 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_3a027f114e1e4d5fabe7ddc8f584e3ec" } new_table_name: "default.Table_77b80258ea294c47bf9a6f1adb45c178" modify_external_catalogs: true
I20260505 07:53:57.745658 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 397d33e908af4a0bb8dde26d273be01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.748171 593 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e]). Expected version 2 got 0
I20260505 07:53:57.749002 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_157815480b75424885451ec833e3438d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.749047 1425 raft_consensus.cc:493] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.749200 1425 raft_consensus.cc:515] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:57.749434 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_157815480b75424885451ec833e3438d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.751870 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_b64f571bfd004207bd2fb763c53b9502"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.752120 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1b6fae561a60438aa62f5edf8eaea6c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.752437 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b64f571bfd004207bd2fb763c53b9502 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:57.752604 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1b6fae561a60438aa62f5edf8eaea6c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.754185 600 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e]). Expected version 2 got 0
I20260505 07:53:57.754480 1425 leader_election.cc:290] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.758177 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 323269e708964848a94c64e6a781a860 with delete_type TABLET_DATA_DELETED (replaced by tablet e010f23c4df647ab863e03c3cfed7898) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:57.758731 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ef2c24cce2148ecb1409ecaa29a8130 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.761679 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ed3f6be0222a4436802966dd49d1dfad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.762569 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed3f6be0222a4436802966dd49d1dfad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:57.765107 988 leader_election.cc:343] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 4d6490f568024510ab843ec5205a4d1a
I20260505 07:53:57.766738 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet d622612bdc404727a27b83769bf39738 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.769413 1364 raft_consensus.cc:2749] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:53:57.771659 991 leader_election.cc:343] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 4d6490f568024510ab843ec5205a4d1a
I20260505 07:53:57.772197 991 leader_election.cc:304] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.772490 1178 ts_tablet_manager.cc:1950] T 772a65f4016843d18fbca25842f76de0 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:57.772645 1364 raft_consensus.cc:2749] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.770044 1470 tablet.cc:1724] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:53:57.773258 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e4e5af08ddc4c77b5640b8295b236b1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:57.780170 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet accb2dedc30e494ebd22c33594cee661 consensus state has no leader
I20260505 07:53:57.782821 1172 tablet_bootstrap.cc:654] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.799316 601 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
I20260505 07:53:57.811928 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 46274a44a06b4306b5dac43c66c85a80
I20260505 07:53:57.812896 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 46274a44a06b4306b5dac43c66c85a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:57.814572 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 46274a44a06b4306b5dac43c66c85a80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46274a44a06b4306b5dac43c66c85a80 already in progress: creating tablet
I20260505 07:53:57.814841 1178 tablet_replica.cc:333] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:57.815024 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 772a65f4016843d18fbca25842f76de0 (table default.Table_4805d11726a3446b86b7a990c0dbb3c5 [id=035a76f8e6f34bc1bb1c75c4908bb4d7]) successfully deleted
I20260505 07:53:57.815738 1178 raft_consensus.cc:2243] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.816022 1178 raft_consensus.cc:2272] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.816289 595 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:53:57.816533 595 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e]). Expected version 2 got 0
I20260505 07:53:57.817520 1178 ts_tablet_manager.cc:1916] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.818758 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d023acedd7c842ebb396d7354c41b490"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.819173 1029 tablet_service.cc:1558] Processing DeleteTablet for tablet 46274a44a06b4306b5dac43c66c85a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:57.819284 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d023acedd7c842ebb396d7354c41b490 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:57.822288 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c822fafe72544ca99fde910c8e764755 consensus state has no leader
I20260505 07:53:57.824122 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_75872f5895d34e0ab0677a846b5b5926" } new_table_name: "default.Table_b7306feea82e49bc917b31d7feb4f1e0" modify_external_catalogs: true
I20260505 07:53:57.825703 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4060b1611196402a829ecc88ff74d21f" } modify_external_catalogs: true
I20260505 07:53:57.825906 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4060b1611196402a829ecc88ff74d21f" } modify_external_catalogs: true
I20260505 07:53:57.827225 594 master_service.cc:954] ReplaceTablet: received request to replace tablet a9494c3b6a99487596a030856b371b68 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.829066 1470 tablet.cc:1724] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:53:57.831357 897 tablet_service.cc:1558] Processing DeleteTablet for tablet 46274a44a06b4306b5dac43c66c85a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:57.832139 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 46274a44a06b4306b5dac43c66c85a80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46274a44a06b4306b5dac43c66c85a80 already in progress: creating tablet
W20260505 07:53:57.833631 585 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 83e62b7af9b847b7a9005e4345905c0d (table default.Table_5fd319488e1d4b9da8f13212d678a934 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 83e62b7af9b847b7a9005e4345905c0d consensus state has no leader
I20260505 07:53:57.835716 1172 tablet_bootstrap.cc:492] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:57.835892 1172 ts_tablet_manager.cc:1403] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.133s user 0.004s sys 0.001s
I20260505 07:53:57.835901 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_786f03e0f3404d69a9f8003f5e62368c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.836514 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_786f03e0f3404d69a9f8003f5e62368c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.836670 1172 raft_consensus.cc:359] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.836882 1172 raft_consensus.cc:385] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.836951 1172 raft_consensus.cc:740] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.839849 1480 tablet.cc:1724] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260505 07:53:57.842481 600 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e] alter complete (version 2)
I20260505 07:53:57.843051 600 master_service.cc:954] ReplaceTablet: received request to replace tablet 003cd4472b424aacbba8c5f89900e228 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.843565 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 5dab02a546b944f799045723a515118f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.843984 1175 tablet_replica.cc:333] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:57.844195 595 master_service.cc:954] ReplaceTablet: received request to replace tablet d3d11bd19f074bbab101448b45426b7a from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.844692 1175 raft_consensus.cc:2243] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.844861 1175 raft_consensus.cc:2272] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.847241 1175 ts_tablet_manager.cc:1916] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.847832 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 5dab02a546b944f799045723a515118f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.848737 1181 tablet_bootstrap.cc:492] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:57.848865 1181 ts_tablet_manager.cc:1403] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.206s user 0.003s sys 0.006s
I20260505 07:53:57.849596 1181 raft_consensus.cc:359] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.849754 1181 raft_consensus.cc:385] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.849841 1181 raft_consensus.cc:740] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.850072 1181 consensus_queue.cc:260] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.850750 1181 ts_tablet_manager.cc:1434] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:53:57.851032 1181 tablet_bootstrap.cc:492] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:57.851269 600 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0ec687e90aea4825b28b36eda22678fe" } modify_external_catalogs: true
I20260505 07:53:57.851419 600 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0ec687e90aea4825b28b36eda22678fe" } modify_external_catalogs: true
I20260505 07:53:57.852651 1181 tablet_bootstrap.cc:654] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.853613 601 master_service.cc:954] ReplaceTablet: received request to replace tablet 909306e1e5f34b7c8fbc0d86cafce531 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.856567 1175 ts_tablet_manager.cc:1929] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.856757 1175 log.cc:1199] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/5dab02a546b944f799045723a515118f
I20260505 07:53:57.857144 1175 ts_tablet_manager.cc:1950] T 5dab02a546b944f799045723a515118f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:57.858299 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 5dab02a546b944f799045723a515118f (table default.Table_2a2802255e0743c18185473d0dcded48 [id=b2adaf7422c7403c8b331c0d33f0ad53]) successfully deleted
I20260505 07:53:57.861595 1172 consensus_queue.cc:260] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.864245 1176 tablet_bootstrap.cc:492] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:57.864423 1176 ts_tablet_manager.cc:1403] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.127s user 0.028s sys 0.034s
I20260505 07:53:57.865154 1176 raft_consensus.cc:359] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.865319 1176 raft_consensus.cc:385] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.865396 1176 raft_consensus.cc:740] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.865638 1176 consensus_queue.cc:260] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.866835 1176 ts_tablet_manager.cc:1434] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:53:57.867138 1176 tablet_bootstrap.cc:492] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:57.870000 829 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33565
I20260505 07:53:57.870764 897 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ac06bb9d273457ead843f5dd754539c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:57.871412 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 3ac06bb9d273457ead843f5dd754539c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3ac06bb9d273457ead843f5dd754539c already in progress: creating tablet
I20260505 07:53:57.874116 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 2782cf753bc44bf4a102dd785ea3c6b8 with delete_type TABLET_DATA_DELETED (replaced by tablet 8d1ce00e487d4702a6affcad57a7fbe1) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:57.874461 1175 tablet_replica.cc:333] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:57.874697 1175 raft_consensus.cc:2243] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.874879 1175 raft_consensus.cc:2272] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.876030 1175 ts_tablet_manager.cc:1916] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.877267 1172 ts_tablet_manager.cc:1434] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.041s user 0.002s sys 0.000s
I20260505 07:53:57.880064 897 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ac06bb9d273457ead843f5dd754539c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.880478 1171 tablet_replica.cc:333] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:57.880640 1171 raft_consensus.cc:2243] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.880800 1171 raft_consensus.cc:2272] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:57.881629 1171 ts_tablet_manager.cc:1916] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:57.886076 1364 raft_consensus.cc:493] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.886240 1364 raft_consensus.cc:515] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:57.886878 1364 leader_election.cc:290] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:53:57.887710 991 leader_election.cc:343] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 956b5e05bda648349ba26d5c1ea3b279
I20260505 07:53:57.890043 1364 raft_consensus.cc:493] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.890188 1364 raft_consensus.cc:515] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.890692 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956b5e05bda648349ba26d5c1ea3b279" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:57.890769 1364 leader_election.cc:290] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:53:57.891319 991 leader_election.cc:343] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 1765a95cc3414c73b7e3918913a35937
I20260505 07:53:57.891491 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 2782cf753bc44bf4a102dd785ea3c6b8 with delete_type TABLET_DATA_DELETED (replaced by tablet 8d1ce00e487d4702a6affcad57a7fbe1) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.891599 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1765a95cc3414c73b7e3918913a35937" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:57.891947 988 leader_election.cc:343] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.892129 988 leader_election.cc:304] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.892405 1364 raft_consensus.cc:2749] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:53:57.892926 988 leader_election.cc:343] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.893070 988 leader_election.cc:304] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.893373 1171 ts_tablet_manager.cc:1929] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.893388 1364 raft_consensus.cc:2749] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.893481 1171 log.cc:1199] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/3ac06bb9d273457ead843f5dd754539c
I20260505 07:53:57.894064 1171 ts_tablet_manager.cc:1950] T 3ac06bb9d273457ead843f5dd754539c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:57.894598 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 3ac06bb9d273457ead843f5dd754539c (table default.Table_4e9260f6bf4c4ea3b35c44a98dcf15f1 [id=4c5b3219388f43d4866a6ef3b32bf603]) successfully deleted
I20260505 07:53:57.894867 1178 ts_tablet_manager.cc:1929] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.894984 1178 log.cc:1199] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/fea61fad19b34ffaa0b9df3a51979b18
I20260505 07:53:57.895336 1178 ts_tablet_manager.cc:1950] T fea61fad19b34ffaa0b9df3a51979b18 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:57.897336 1172 tablet_bootstrap.cc:492] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:57.900797 1445 mvcc.cc:204] Tried to move back new op lower bound from 7282555440790614016 to 7282555435750510592. Current Snapshot: MvccSnapshot[applied={T|T < 7282555440790614016}]
I20260505 07:53:57.901181 920 raft_consensus.cc:1275] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Refusing update from remote peer d63a3e535f8e4dd4a88e447253405b77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260505 07:53:57.901219 781 raft_consensus.cc:3060] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:57.901407 1364 raft_consensus.cc:493] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.901552 1364 raft_consensus.cc:515] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.902077 1364 leader_election.cc:290] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.902216 1364 consensus_queue.cc:1048] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:53:57.902856 991 leader_election.cc:343] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: ce9d01b0b5464a65966f26839b48aaf1
I20260505 07:53:57.904418 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce9d01b0b5464a65966f26839b48aaf1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:57.905028 988 leader_election.cc:343] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.905179 988 leader_election.cc:304] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:57.905441 1425 raft_consensus.cc:2749] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:57.905820 1176 tablet_bootstrap.cc:654] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.906157 781 raft_consensus.cc:1275] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Refusing update from remote peer d63a3e535f8e4dd4a88e447253405b77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260505 07:53:57.906901 596 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]). Expected version 3 got 0
I20260505 07:53:57.906869 1425 consensus_queue.cc:1048] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:53:57.908962 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 297537a92684425d8508f7b3b4c9d379 consensus state has no leader
I20260505 07:53:57.910676 1175 ts_tablet_manager.cc:1929] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:57.910799 1175 log.cc:1199] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/2782cf753bc44bf4a102dd785ea3c6b8
I20260505 07:53:57.911139 1175 ts_tablet_manager.cc:1950] T 2782cf753bc44bf4a102dd785ea3c6b8 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:57.913687 1102 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33565
I20260505 07:53:57.916639 1020 tablet_service.cc:1511] Processing CreateTablet for tablet 90a13ca612f241fc97ea12124677a0b1 (DEFAULT_TABLE table=default.Table_09814f6166bb4e6e9978387328078099 [id=50162ab861e0468d8622e6d2f33d9e79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.917335 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet fea61fad19b34ffaa0b9df3a51979b18 (table default.Table_633da5b39da34f5c88163c9936fd0212 [id=d53c653124364ea5acc357977c9db864]) successfully deleted
I20260505 07:53:57.917429 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 (DEFAULT_TABLE table=default.Table_e9ba8a72d90d4a3da2c06b684fe35033 [id=5c8387e5e7f84731afc62bbea6475a85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.919736 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 2782cf753bc44bf4a102dd785ea3c6b8 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]) successfully deleted
I20260505 07:53:57.921748 746 tablet_service.cc:1511] Processing CreateTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 (DEFAULT_TABLE table=default.Table_e9ba8a72d90d4a3da2c06b684fe35033 [id=5c8387e5e7f84731afc62bbea6475a85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.922125 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1602ccbae24544c4a6cf14ea073880b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.927440 1366 raft_consensus.cc:493] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.927692 1366 raft_consensus.cc:515] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:57.927938 761 tablet_service.cc:1511] Processing CreateTablet for tablet 4cb7b3fded5d4def8db8d5c1e77ddcda (DEFAULT_TABLE table=default.Table_cc284917d2aa41d9956affce2c946825 [id=231630aa148e43d1839edb43f686ad20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.928361 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb7b3fded5d4def8db8d5c1e77ddcda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.928495 1366 leader_election.cc:290] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:57.928784 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b404e50ae092419c9396772b3b14eeaa" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:57.929399 854 leader_election.cc:343] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.930684 1176 tablet_bootstrap.cc:492] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:57.930873 1176 ts_tablet_manager.cc:1403] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.064s user 0.003s sys 0.003s
I20260505 07:53:57.931936 754 tablet_service.cc:1511] Processing CreateTablet for tablet 2ca12a17fb014374bf7ed764a91e6985 (DEFAULT_TABLE table=default.Table_f3a7ead203124b49a437c53941ec4d58 [id=8ebfbe0aa09a43bdbb4b8f323658b3b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.932308 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ca12a17fb014374bf7ed764a91e6985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.935696 749 tablet_service.cc:1511] Processing CreateTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 (DEFAULT_TABLE table=default.Table_74517dd9ec674932b68437af722ccff7 [id=169b57b0e527456cb435bd97c47782bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.935995 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0b9e8eb767a4eceb583ff9b47000d58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.939960 750 tablet_service.cc:1511] Processing CreateTablet for tablet 5ad8a79245ad47738d079752e51aed2b (DEFAULT_TABLE table=default.Table_ba3de8258d7e46c1ad3a509b8e00b806 [id=75d1988e3a9e4ff3a8c206a662f55131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.940342 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad8a79245ad47738d079752e51aed2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.942212 1366 raft_consensus.cc:493] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.942364 1366 raft_consensus.cc:515] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:57.943034 1366 leader_election.cc:290] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:57.943492 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571257bbb7e94315b60d84d6aa4917b1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:57.944027 854 leader_election.cc:343] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:57.944912 1176 raft_consensus.cc:359] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:57.945137 1176 raft_consensus.cc:385] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:57.945220 1176 raft_consensus.cc:740] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.945461 1176 consensus_queue.cc:260] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:57.945668 762 tablet_service.cc:1511] Processing CreateTablet for tablet 90a13ca612f241fc97ea12124677a0b1 (DEFAULT_TABLE table=default.Table_09814f6166bb4e6e9978387328078099 [id=50162ab861e0468d8622e6d2f33d9e79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.946010 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a13ca612f241fc97ea12124677a0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.947332 1176 ts_tablet_manager.cc:1434] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.016s user 0.001s sys 0.002s
I20260505 07:53:57.947809 1176 tablet_bootstrap.cc:492] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:57.948501 598 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]). Expected version 3 got 0
I20260505 07:53:57.939512 758 tablet_service.cc:1511] Processing CreateTablet for tablet e79a1da502604d05a62e27f575b8b8f4 (DEFAULT_TABLE table=default.Table_ab5be55234c742aea9421c70e2dad3bb [id=70c9d1bf26e34dd69abefcef94c785c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.949401 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e79a1da502604d05a62e27f575b8b8f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.950104 1176 tablet_bootstrap.cc:654] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.952682 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4011b7c55a4e4e149f8aa0b6f2981824" } modify_external_catalogs: true
I20260505 07:53:57.952912 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4011b7c55a4e4e149f8aa0b6f2981824" } modify_external_catalogs: true
I20260505 07:53:57.953025 596 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f95053c53c17434b9175534e6d3ac45e" } new_table_name: "default.Table_3a76cf09eaed4c9a8c267714a164dd66" modify_external_catalogs: true
I20260505 07:53:57.954264 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_73ba90aede024514abb230661a44a4ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:57.954416 599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_5a41e60184ab44a99342444292f0cc5f" } modify_external_catalogs: true
I20260505 07:53:57.954599 599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_5a41e60184ab44a99342444292f0cc5f" } modify_external_catalogs: true
W20260505 07:53:57.954809 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73ba90aede024514abb230661a44a4ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.957224 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_cb21f88a808c477b8a55a83e7507b951" } new_table_name: "default.Table_268b9129e7fb4560bcbbb7e5427975fa" modify_external_catalogs: true
I20260505 07:53:57.958976 600 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 141665fcd59048bf926acd0287f1853e
I20260505 07:53:57.959827 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_8564f8e67e6f4995b312673dd0d63f52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:57.960325 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8564f8e67e6f4995b312673dd0d63f52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:57.961313 1467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Bootstrap replayed 1/1 log segments. Stats: ops{read=22 overwritten=0 applied=22 ignored=0} inserts{seen=23 ignored=0} mutations{seen=19 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260505 07:53:57.962620 1467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Bootstrap complete.
I20260505 07:53:57.965687 1379 raft_consensus.cc:493] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:57.965828 1379 raft_consensus.cc:515] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:57.966401 1379 leader_election.cc:290] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:57.969162 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 141665fcd59048bf926acd0287f1853e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:57.971185 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 141665fcd59048bf926acd0287f1853e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 141665fcd59048bf926acd0287f1853e already in progress: creating tablet
I20260505 07:53:57.971354 1172 tablet_bootstrap.cc:654] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:57.970978 1467 raft_consensus.cc:359] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:57.971980 1467 raft_consensus.cc:740] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23c57a2c927c4c54bccecfeb3a775311, State: Initialized, Role: FOLLOWER
I20260505 07:53:57.972817 897 tablet_service.cc:1511] Processing CreateTablet for tablet 90a13ca612f241fc97ea12124677a0b1 (DEFAULT_TABLE table=default.Table_09814f6166bb4e6e9978387328078099 [id=50162ab861e0468d8622e6d2f33d9e79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.973196 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a13ca612f241fc97ea12124677a0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.973654 886 tablet_service.cc:1511] Processing CreateTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 (DEFAULT_TABLE table=default.Table_e9ba8a72d90d4a3da2c06b684fe35033 [id=5c8387e5e7f84731afc62bbea6475a85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.973862 1467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 22, Last appended: 1.22, Last appended by leader: 22, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:57.974035 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1602ccbae24544c4a6cf14ea073880b4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:57.975811 718 leader_election.cc:343] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 956b5e05bda648349ba26d5c1ea3b279
I20260505 07:53:57.975849 882 tablet_service.cc:1511] Processing CreateTablet for tablet 4cb7b3fded5d4def8db8d5c1e77ddcda (DEFAULT_TABLE table=default.Table_cc284917d2aa41d9956affce2c946825 [id=231630aa148e43d1839edb43f686ad20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.976204 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb7b3fded5d4def8db8d5c1e77ddcda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.979837 883 tablet_service.cc:1511] Processing CreateTablet for tablet 2ca12a17fb014374bf7ed764a91e6985 (DEFAULT_TABLE table=default.Table_f3a7ead203124b49a437c53941ec4d58 [id=8ebfbe0aa09a43bdbb4b8f323658b3b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.980145 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ca12a17fb014374bf7ed764a91e6985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.980361 971 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33565
I20260505 07:53:57.982825 899 tablet_service.cc:1511] Processing CreateTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 (DEFAULT_TABLE table=default.Table_74517dd9ec674932b68437af722ccff7 [id=169b57b0e527456cb435bd97c47782bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.983225 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0b9e8eb767a4eceb583ff9b47000d58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:57.983100 1490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:57.983332 1490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:53:57.985649 890 tablet_service.cc:1511] Processing CreateTablet for tablet e79a1da502604d05a62e27f575b8b8f4 (DEFAULT_TABLE table=default.Table_ab5be55234c742aea9421c70e2dad3bb [id=70c9d1bf26e34dd69abefcef94c785c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.986176 1467 sys_catalog.cc:565] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: configured and running, proceeding with master startup.
I20260505 07:53:57.989039 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet a9494c3b6a99487596a030856b371b68
I20260505 07:53:57.989693 594 catalog_manager.cc:6741] ReplaceTablet: tablet a9494c3b6a99487596a030856b371b68 deleted and replaced by tablet 6fe19306f7df49fdb771d21f6560e697
I20260505 07:53:57.989935 594 master_service.cc:954] ReplaceTablet: received request to replace tablet 872362aecf6a44b59e586b1fa7637c8b from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:57.990442 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet d3d11bd19f074bbab101448b45426b7a
I20260505 07:53:57.990921 595 catalog_manager.cc:6741] ReplaceTablet: tablet d3d11bd19f074bbab101448b45426b7a deleted and replaced by tablet 76939b7978ee4f7ea0ad29312de2a31a
I20260505 07:53:57.991710 1024 tablet_service.cc:1511] Processing CreateTablet for tablet e79a1da502604d05a62e27f575b8b8f4 (DEFAULT_TABLE table=default.Table_ab5be55234c742aea9421c70e2dad3bb [id=70c9d1bf26e34dd69abefcef94c785c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.991940 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 2ca12a17fb014374bf7ed764a91e6985 (DEFAULT_TABLE table=default.Table_f3a7ead203124b49a437c53941ec4d58 [id=8ebfbe0aa09a43bdbb4b8f323658b3b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:57.992516 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet bc9ae6404dd64bd8bf4898768e94e4f1
I20260505 07:53:57.994625 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 141665fcd59048bf926acd0287f1853e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:57.994740 887 tablet_service.cc:1558] Processing DeleteTablet for tablet a9494c3b6a99487596a030856b371b68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6fe19306f7df49fdb771d21f6560e697) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.995191 893 tablet_service.cc:1558] Processing DeleteTablet for tablet bc9ae6404dd64bd8bf4898768e94e4f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.995312 885 tablet_service.cc:1558] Processing DeleteTablet for tablet d3d11bd19f074bbab101448b45426b7a with delete_type TABLET_DATA_DELETED (replaced by tablet 76939b7978ee4f7ea0ad29312de2a31a) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.994992 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 141665fcd59048bf926acd0287f1853e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:57.997460 1364 raft_consensus.cc:493] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:53:57.997764 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a9494c3b6a99487596a030856b371b68 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9494c3b6a99487596a030856b371b68 already in progress: creating tablet
W20260505 07:53:57.998009 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet bc9ae6404dd64bd8bf4898768e94e4f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc9ae6404dd64bd8bf4898768e94e4f1 already in progress: creating tablet
W20260505 07:53:57.998217 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet d3d11bd19f074bbab101448b45426b7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3d11bd19f074bbab101448b45426b7a already in progress: creating tablet
I20260505 07:53:57.999820 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e79a1da502604d05a62e27f575b8b8f4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:58.001772 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 141665fcd59048bf926acd0287f1853e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 141665fcd59048bf926acd0287f1853e already in progress: creating tablet
I20260505 07:53:58.005071 891 tablet_service.cc:1511] Processing CreateTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c (DEFAULT_TABLE table=default.Table_f95053c53c17434b9175534e6d3ac45e [id=6be00fd9040d4febaef6b16dec369a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.005412 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f121fdd001ec4ea38ccd24a6afd9e51c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.006534 1425 raft_consensus.cc:493] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:58.006707 882 tablet_service.cc:1511] Processing CreateTablet for tablet 5ad8a79245ad47738d079752e51aed2b (DEFAULT_TABLE table=default.Table_ba3de8258d7e46c1ad3a509b8e00b806 [id=75d1988e3a9e4ff3a8c206a662f55131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.007203 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad8a79245ad47738d079752e51aed2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.008023 762 tablet_service.cc:1511] Processing CreateTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c (DEFAULT_TABLE table=default.Table_f95053c53c17434b9175534e6d3ac45e [id=6be00fd9040d4febaef6b16dec369a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.008700 885 tablet_service.cc:1511] Processing CreateTablet for tablet d442119970a345f28de029eb2139f5c1 (DEFAULT_TABLE table=default.Table_e7ad0f00a4fc456e944f70e21d505fd5 [id=d5242964630749eeb15c3c7c9b0f3686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.009089 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d442119970a345f28de029eb2139f5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.009585 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1602ccbae24544c4a6cf14ea073880b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.009889 1178 tablet_replica.cc:333] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:58.010113 1178 raft_consensus.cc:2243] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:57.998021 1029 tablet_service.cc:1511] Processing CreateTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 (DEFAULT_TABLE table=default.Table_74517dd9ec674932b68437af722ccff7 [id=169b57b0e527456cb435bd97c47782bb]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:57.997396 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 44216.3us
I20260505 07:53:58.010970 897 tablet_service.cc:1511] Processing CreateTablet for tablet feeafd614b3e4281a8e45b950e699b63 (DEFAULT_TABLE table=default.Table_7eb9890f5e7349748b35f1134174580e [id=99a81018fe1b45638089d579b76029d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.008589 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f121fdd001ec4ea38ccd24a6afd9e51c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.011382 1178 raft_consensus.cc:2272] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.011745 1459 tablet.cc:1724] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:53:58.013803 754 tablet_service.cc:1511] Processing CreateTablet for tablet feeafd614b3e4281a8e45b950e699b63 (DEFAULT_TABLE table=default.Table_7eb9890f5e7349748b35f1134174580e [id=99a81018fe1b45638089d579b76029d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.014209 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b404e50ae092419c9396772b3b14eeaa" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:58.011266 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet feeafd614b3e4281a8e45b950e699b63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.016374 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet feeafd614b3e4281a8e45b950e699b63. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:58.016618 856 leader_election.cc:343] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:58.016772 856 leader_election.cc:304] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:58.017030 1366 raft_consensus.cc:2749] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:58.018998 1027 tablet_service.cc:1511] Processing CreateTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c (DEFAULT_TABLE table=default.Table_f95053c53c17434b9175534e6d3ac45e [id=6be00fd9040d4febaef6b16dec369a3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.019496 761 tablet_service.cc:1558] Processing DeleteTablet for tablet bc9ae6404dd64bd8bf4898768e94e4f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.019779 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet d3d11bd19f074bbab101448b45426b7a with delete_type TABLET_DATA_DELETED (replaced by tablet 76939b7978ee4f7ea0ad29312de2a31a) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.020502 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571257bbb7e94315b60d84d6aa4917b1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:58.021036 1425 raft_consensus.cc:515] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.021705 1425 leader_election.cc:290] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:58.021744 758 tablet_service.cc:1558] Processing DeleteTablet for tablet a9494c3b6a99487596a030856b371b68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6fe19306f7df49fdb771d21f6560e697) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.021139 1181 tablet_bootstrap.cc:492] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
W20260505 07:53:58.023150 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet bc9ae6404dd64bd8bf4898768e94e4f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc9ae6404dd64bd8bf4898768e94e4f1 already in progress: creating tablet
I20260505 07:53:58.023507 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet a9494c3b6a99487596a030856b371b68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6fe19306f7df49fdb771d21f6560e697) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.025876 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 5ad8a79245ad47738d079752e51aed2b (DEFAULT_TABLE table=default.Table_ba3de8258d7e46c1ad3a509b8e00b806 [id=75d1988e3a9e4ff3a8c206a662f55131]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.026433 1178 ts_tablet_manager.cc:1916] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:58.027375 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet a9494c3b6a99487596a030856b371b68 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9494c3b6a99487596a030856b371b68 already in progress: creating tablet
I20260505 07:53:58.027617 750 tablet_service.cc:1511] Processing CreateTablet for tablet d442119970a345f28de029eb2139f5c1 (DEFAULT_TABLE table=default.Table_e7ad0f00a4fc456e944f70e21d505fd5 [id=d5242964630749eeb15c3c7c9b0f3686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.028425 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d442119970a345f28de029eb2139f5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.031364 1364 raft_consensus.cc:515] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:58.032106 749 tablet_service.cc:1558] Processing DeleteTablet for tablet d3d11bd19f074bbab101448b45426b7a with delete_type TABLET_DATA_DELETED (replaced by tablet 76939b7978ee4f7ea0ad29312de2a31a) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:58.032786 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet d3d11bd19f074bbab101448b45426b7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3d11bd19f074bbab101448b45426b7a already in progress: creating tablet
W20260505 07:53:58.035733 856 leader_election.cc:343] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:58.035847 856 leader_election.cc:304] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:58.036056 1366 raft_consensus.cc:2749] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:58.037961 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956b5e05bda648349ba26d5c1ea3b279" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:58.038440 1035 tablet_service.cc:1511] Processing CreateTablet for tablet feeafd614b3e4281a8e45b950e699b63 (DEFAULT_TABLE table=default.Table_7eb9890f5e7349748b35f1134174580e [id=99a81018fe1b45638089d579b76029d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.040657 749 tablet_service.cc:1511] Processing CreateTablet for tablet 727cacd242844455915bd9963d2db19f (DEFAULT_TABLE table=default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.041348 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 727cacd242844455915bd9963d2db19f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.042603 758 tablet_service.cc:1511] Processing CreateTablet for tablet 3a73dbb979344805876efbafbf1b2985 (DEFAULT_TABLE table=default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5 [id=c10b484439c443368141ceb4cf43365f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.043085 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a73dbb979344805876efbafbf1b2985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.045976 886 tablet_service.cc:1511] Processing CreateTablet for tablet 3a73dbb979344805876efbafbf1b2985 (DEFAULT_TABLE table=default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5 [id=c10b484439c443368141ceb4cf43365f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.046129 971 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33565
I20260505 07:53:58.046384 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a73dbb979344805876efbafbf1b2985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.053092 601 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 909306e1e5f34b7c8fbc0d86cafce531
I20260505 07:53:58.053530 601 catalog_manager.cc:6741] ReplaceTablet: tablet 909306e1e5f34b7c8fbc0d86cafce531 deleted and replaced by tablet 66082a7a75e345639e7a268727922fbb
W20260505 07:53:58.055509 596 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f121fdd001ec4ea38ccd24a6afd9e51c consensus state has no leader
I20260505 07:53:58.057708 1055 raft_consensus.cc:2468] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:53:58.058209 717 leader_election.cc:304] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:53:58.031260 1483 tablet.cc:1724] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:53:58.058684 1379 raft_consensus.cc:2804] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:58.058785 1379 raft_consensus.cc:493] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:58.058880 1379 raft_consensus.cc:3060] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:58.058826 1026 tablet_service.cc:1511] Processing CreateTablet for tablet d442119970a345f28de029eb2139f5c1 (DEFAULT_TABLE table=default.Table_e7ad0f00a4fc456e944f70e21d505fd5 [id=d5242964630749eeb15c3c7c9b0f3686]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.059145 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 4cb7b3fded5d4def8db8d5c1e77ddcda (DEFAULT_TABLE table=default.Table_cc284917d2aa41d9956affce2c946825 [id=231630aa148e43d1839edb43f686ad20]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.060640 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 3a73dbb979344805876efbafbf1b2985 (DEFAULT_TABLE table=default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5 [id=c10b484439c443368141ceb4cf43365f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.060890 1364 leader_election.cc:290] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:58.061193 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet bc9ae6404dd64bd8bf4898768e94e4f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.061347 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 727cacd242844455915bd9963d2db19f (DEFAULT_TABLE table=default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.061384 1379 raft_consensus.cc:515] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:58.062312 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90a13ca612f241fc97ea12124677a0b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.062762 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ca12a17fb014374bf7ed764a91e6985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.063609 885 tablet_service.cc:1511] Processing CreateTablet for tablet 727cacd242844455915bd9963d2db19f (DEFAULT_TABLE table=default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.063966 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 727cacd242844455915bd9963d2db19f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.064406 1379 leader_election.cc:290] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:58.067688 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 with delete_type TABLET_DATA_DELETED (replaced by tablet 66082a7a75e345639e7a268727922fbb) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:58.068729 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 909306e1e5f34b7c8fbc0d86cafce531 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 909306e1e5f34b7c8fbc0d86cafce531 already in progress: creating tablet
W20260505 07:53:58.072989 991 leader_election.cc:343] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 2782cf753bc44bf4a102dd785ea3c6b8
I20260505 07:53:58.073055 1181 ts_tablet_manager.cc:1403] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.171s user 0.017s sys 0.011s
I20260505 07:53:58.073077 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 with delete_type TABLET_DATA_DELETED (replaced by tablet 66082a7a75e345639e7a268727922fbb) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:58.073173 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 50840.1us
I20260505 07:53:58.074223 1181 raft_consensus.cc:359] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:58.074061 991 leader_election.cc:343] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 5f57568e1c4c4573ac18d9f621eeb4c7
I20260505 07:53:58.077134 1181 raft_consensus.cc:385] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.077267 1181 raft_consensus.cc:740] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.078382 1172 tablet_bootstrap.cc:492] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:58.079170 1172 ts_tablet_manager.cc:1403] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.182s user 0.007s sys 0.007s
I20260505 07:53:58.082047 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1b644b185b194940a8a8a90595ae9e90" } modify_external_catalogs: true
I20260505 07:53:58.082264 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1b644b185b194940a8a8a90595ae9e90" } modify_external_catalogs: true
I20260505 07:53:58.084138 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_55f3a1ca8c53455993b486b0798f0425"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.084715 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55f3a1ca8c53455993b486b0798f0425 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.085454 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_9912957f51d44c0ab0b2aaaf516ad36f" } new_table_name: "default.Table_394242d5ee184ae2b22cc355a2160fb9" modify_external_catalogs: true
I20260505 07:53:58.092017 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f121fdd001ec4ea38ccd24a6afd9e51c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.092541 1181 consensus_queue.cc:260] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.093241 1181 ts_tablet_manager.cc:1434] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20260505 07:53:58.093888 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "956b5e05bda648349ba26d5c1ea3b279" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:53:58.094132 1055 raft_consensus.cc:3060] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:53:58.094867 988 leader_election.cc:343] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 2782cf753bc44bf4a102dd785ea3c6b8
I20260505 07:53:58.095600 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ad8a79245ad47738d079752e51aed2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.096124 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 727cacd242844455915bd9963d2db19f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.096145 988 leader_election.cc:304] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:53:58.097656 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 66337.9us
I20260505 07:53:58.093585 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 with delete_type TABLET_DATA_DELETED (replaced by tablet 66082a7a75e345639e7a268727922fbb) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:58.098875 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 909306e1e5f34b7c8fbc0d86cafce531 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 909306e1e5f34b7c8fbc0d86cafce531 already in progress: creating tablet
I20260505 07:53:58.098935 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 5de9e0fe8eb7477ab1cfa64b808575d8
I20260505 07:53:58.099704 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c41ba7470a784645bdddd3a262d4bb4e" } new_table_name: "default.Table_1523b3ae7cd94c599e60a187bc677fc0" modify_external_catalogs: true
I20260505 07:53:58.100329 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.100649 1175 tablet_replica.cc:333] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:58.100848 1175 raft_consensus.cc:2243] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:53:58.101197 1175 raft_consensus.cc:2272] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.101807 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_c22dca26148c48f882eeea850f50bea4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:58.101977 1175 ts_tablet_manager.cc:1916] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:58.102280 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c22dca26148c48f882eeea850f50bea4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.108027 1364 raft_consensus.cc:2749] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:53:58.109150 988 leader_election.cc:343] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 5f57568e1c4c4573ac18d9f621eeb4c7
I20260505 07:53:58.113484 599 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet a625fe1119274d0eb77cca6ee07720d0
W20260505 07:53:58.114634 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62bf3a6bc0ac42a99a4181f820e3ca87 consensus state has no leader
I20260505 07:53:58.114550 1175 ts_tablet_manager.cc:1929] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:53:58.115350 1175 log.cc:1199] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/5de9e0fe8eb7477ab1cfa64b808575d8
I20260505 07:53:58.118713 1175 ts_tablet_manager.cc:1950] T 5de9e0fe8eb7477ab1cfa64b808575d8 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:58.127476 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e]) successfully deleted
I20260505 07:53:58.131747 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0b9e8eb767a4eceb583ff9b47000d58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.136620 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet ff7ed926a3ff4e7993864de734ca3c50
I20260505 07:53:58.137169 593 master_service.cc:954] ReplaceTablet: received request to replace tablet bf76e868bbcb44b0b1887c84dd27f093 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:58.137166 594 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]). Expected version 3 got 0
I20260505 07:53:58.138490 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7f7ad7a477f249319d0428cd775dc541"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.139164 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7f7ad7a477f249319d0428cd775dc541 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.140734 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_4e8ee4d80a174660827b921c9f211ac4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:58.140831 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_4a5f2bded41d47d59bd0da9575fb955a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.141219 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4a5f2bded41d47d59bd0da9575fb955a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:58.141220 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e8ee4d80a174660827b921c9f211ac4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.141654 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_4d333714e70b4e53b695681972d14faf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.142194 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d333714e70b4e53b695681972d14faf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:58.143610 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 85d485ae2d1d4d76a6a5d988cec8291a consensus state has no leader
I20260505 07:53:58.143954 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6121e74274264813a8cc8dd21d0d5e17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.144500 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6121e74274264813a8cc8dd21d0d5e17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:58.146111 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9044763bed1a41739ecce4def4cca1fe consensus state has no leader
I20260505 07:53:58.146469 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_74517dd9ec674932b68437af722ccff7" } new_table_name: "default.Table_49c6ff5135b14b6b8437bb753e1b6342" modify_external_catalogs: true
I20260505 07:53:58.147975 601 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9] alter complete (version 3)
I20260505 07:53:58.149348 1181 tablet_bootstrap.cc:492] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:58.151343 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_881e42cce93c46cb8c6b2ff28d713b85"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.151950 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_881e42cce93c46cb8c6b2ff28d713b85 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.153553 988 leader_election.cc:304] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:58.157127 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d442119970a345f28de029eb2139f5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.157275 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.157727 1364 raft_consensus.cc:2749] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:58.157810 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb7b3fded5d4def8db8d5c1e77ddcda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.158749 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a73dbb979344805876efbafbf1b2985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.161314 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ba3de8258d7e46c1ad3a509b8e00b806" } modify_external_catalogs: true
I20260505 07:53:58.161501 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ba3de8258d7e46c1ad3a509b8e00b806" } modify_external_catalogs: true
I20260505 07:53:58.162454 762 tablet_service.cc:1558] Processing DeleteTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.162813 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet feeafd614b3e4281a8e45b950e699b63. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:58.163007 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 54357us
I20260505 07:53:58.163878 1176 tablet_bootstrap.cc:492] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:58.164019 1176 ts_tablet_manager.cc:1403] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.216s user 0.005s sys 0.002s
I20260505 07:53:58.164530 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 956b5e05bda648349ba26d5c1ea3b279 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.164880 1465 tablet.cc:1724] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:53:58.164958 1176 raft_consensus.cc:359] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.165161 1176 raft_consensus.cc:385] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.165266 1176 raft_consensus.cc:740] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.165793 1175 tablet_replica.cc:333] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:58.165838 1176 consensus_queue.cc:260] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.165969 1175 raft_consensus.cc:2243] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.166143 1175 raft_consensus.cc:2272] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.167362 601 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]). Expected version 3 got 0
I20260505 07:53:58.170037 1175 ts_tablet_manager.cc:1916] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.175813 1176 ts_tablet_manager.cc:1434] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.012s user 0.000s sys 0.002s
I20260505 07:53:58.181048 1176 tablet_bootstrap.cc:492] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:58.182158 1175 ts_tablet_manager.cc:1929] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.182340 1175 log.cc:1199] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/956b5e05bda648349ba26d5c1ea3b279
I20260505 07:53:58.184514 1175 ts_tablet_manager.cc:1950] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:58.186060 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet bf76e868bbcb44b0b1887c84dd27f093
I20260505 07:53:58.186564 593 catalog_manager.cc:6741] ReplaceTablet: tablet bf76e868bbcb44b0b1887c84dd27f093 deleted and replaced by tablet c837877946604ef29f95dac9dd449bf3
W20260505 07:53:58.190126 718 leader_election.cc:343] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 956b5e05bda648349ba26d5c1ea3b279
I20260505 07:53:58.190680 897 tablet_service.cc:1558] Processing DeleteTablet for tablet 5de9e0fe8eb7477ab1cfa64b808575d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.190918 1171 tablet_replica.cc:333] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:58.191465 1171 raft_consensus.cc:2243] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.191938 1171 raft_consensus.cc:2272] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.192140 899 tablet_service.cc:1558] Processing DeleteTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.192241 891 tablet_service.cc:1558] Processing DeleteTablet for tablet bf76e868bbcb44b0b1887c84dd27f093 with delete_type TABLET_DATA_DELETED (replaced by tablet c837877946604ef29f95dac9dd449bf3) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.193325 1171 ts_tablet_manager.cc:1916] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.194784 1181 tablet_bootstrap.cc:654] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.195981 758 tablet_service.cc:1558] Processing DeleteTablet for tablet ce9d01b0b5464a65966f26839b48aaf1 with delete_type TABLET_DATA_DELETED (replaced by tablet 17327cee69a24e0cb493ff55f948bf6c) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.197130 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e79a1da502604d05a62e27f575b8b8f4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:58.196126 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet ff7ed926a3ff4e7993864de734ca3c50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ff7ed926a3ff4e7993864de734ca3c50 already in progress: creating tablet
I20260505 07:53:58.200485 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.201124 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 56cbbb89f6b84407b2c344ec62402b03 with delete_type TABLET_DATA_DELETED (replaced by tablet 9044763bed1a41739ecce4def4cca1fe) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.203804 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 956b5e05bda648349ba26d5c1ea3b279 (table default.Table_017976a3a6af42c9a9fc722661286c0e [id=d524458d873a48cb980e2f85490232d6]) successfully deleted
I20260505 07:53:58.204636 758 tablet_service.cc:1558] Processing DeleteTablet for tablet bf76e868bbcb44b0b1887c84dd27f093 with delete_type TABLET_DATA_DELETED (replaced by tablet c837877946604ef29f95dac9dd449bf3) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.208773 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet bf76e868bbcb44b0b1887c84dd27f093 with delete_type TABLET_DATA_DELETED (replaced by tablet c837877946604ef29f95dac9dd449bf3) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.216499 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_e9ba8a72d90d4a3da2c06b684fe35033" } modify_external_catalogs: true
I20260505 07:53:58.216779 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_e9ba8a72d90d4a3da2c06b684fe35033" } modify_external_catalogs: true
I20260505 07:53:58.222468 1055 raft_consensus.cc:2468] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:53:58.222994 717 leader_election.cc:304] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
W20260505 07:53:58.223237 717 raft_consensus.cc:2686] T 956b5e05bda648349ba26d5c1ea3b279 P eac7e1e8caba43cdb4df80cbb80e891c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260505 07:53:58.226445 1172 raft_consensus.cc:359] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.226790 601 master_service.cc:954] ReplaceTablet: received request to replace tablet 4258df74ea0841449846cdf1f6e00b9b from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:58.227229 1172 raft_consensus.cc:385] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.227528 1172 raft_consensus.cc:740] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.227808 1172 consensus_queue.cc:260] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.232107 1172 ts_tablet_manager.cc:1434] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.153s user 0.000s sys 0.002s
I20260505 07:53:58.232429 1172 tablet_bootstrap.cc:492] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:58.235982 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a79578757824bfc9df129e3475e7af3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.245792 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 976359601ad74b149696dc7f555b550c from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:58.247421 1171 ts_tablet_manager.cc:1929] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:53:58.247656 1171 log.cc:1199] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/5de9e0fe8eb7477ab1cfa64b808575d8
I20260505 07:53:58.248136 1171 ts_tablet_manager.cc:1950] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:58.248813 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e]) successfully deleted
I20260505 07:53:58.249061 1178 ts_tablet_manager.cc:1929] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.257416 1172 tablet_bootstrap.cc:654] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.259970 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_5d02989a34e8412b819674082ae217f7" } new_table_name: "default.Table_f7876fb512074cf88d88f99db4ae1720" modify_external_catalogs: true
I20260505 07:53:58.262598 1178 log.cc:1199] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/09a8dbb7fe4b406d804c0b62ba006adb
W20260505 07:53:58.264107 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet ff7ed926a3ff4e7993864de734ca3c50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ff7ed926a3ff4e7993864de734ca3c50 already in progress: creating tablet
I20260505 07:53:58.264601 1425 raft_consensus.cc:493] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:53:58.264756 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet bf76e868bbcb44b0b1887c84dd27f093 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf76e868bbcb44b0b1887c84dd27f093 already in progress: creating tablet
I20260505 07:53:58.264752 1425 raft_consensus.cc:515] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.265025 1171 tablet_replica.cc:333] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:58.265189 1171 raft_consensus.cc:2243] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.265362 1171 raft_consensus.cc:2272] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.267130 1425 leader_election.cc:290] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:53:58.267424 991 leader_election.cc:343] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 56cbbb89f6b84407b2c344ec62402b03
W20260505 07:53:58.268658 988 leader_election.cc:343] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 56cbbb89f6b84407b2c344ec62402b03
I20260505 07:53:58.268774 988 leader_election.cc:304] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:58.269182 1425 raft_consensus.cc:2749] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:53:58.274145 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b0b9e8eb767a4eceb583ff9b47000d58 consensus state has no leader
I20260505 07:53:58.276204 891 tablet_service.cc:1511] Processing CreateTablet for tablet e1a403e4708e4725899c1cd5b9fb0e8b (DEFAULT_TABLE table=default.Table_8564f8e67e6f4995b312673dd0d63f52 [id=de1506e299994d1fbe31033d301d4533]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.277148 899 tablet_service.cc:1511] Processing CreateTablet for tablet 17b4e886bebc4ab6a3c95290d18b4229 (DEFAULT_TABLE table=default.Table_786f03e0f3404d69a9f8003f5e62368c [id=a6476fbd5ce8492e8d4e54238c5c019f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.278136 1178 ts_tablet_manager.cc:1950] T 09a8dbb7fe4b406d804c0b62ba006adb P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:58.278914 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 236) took 1053 ms. Trace:
I20260505 07:53:58.279832 1178 rpcz_store.cc:276] 0505 07:53:57.225406 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.225706 (+ 300us) service_pool.cc:224] Handling call
0505 07:53:57.916198 (+690492us) ts_tablet_manager.cc:1151] Deleting tablet 09a8dbb7fe4b406d804c0b62ba006adb
0505 07:53:57.958921 (+ 42723us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:58.217840 (+258919us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.222175 (+ 4335us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.278883 (+ 56708us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":686298}
I20260505 07:53:58.281895 758 tablet_service.cc:1511] Processing CreateTablet for tablet e1a403e4708e4725899c1cd5b9fb0e8b (DEFAULT_TABLE table=default.Table_8564f8e67e6f4995b312673dd0d63f52 [id=de1506e299994d1fbe31033d301d4533]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.282305 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a403e4708e4725899c1cd5b9fb0e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.282246 885 tablet_service.cc:1511] Processing CreateTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 (DEFAULT_TABLE table=default.Table_2717154d3561495692818d7755d53c41 [id=23c6264859f64b598af609ed2fff6e8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.287878 762 tablet_service.cc:1511] Processing CreateTablet for tablet 17b4e886bebc4ab6a3c95290d18b4229 (DEFAULT_TABLE table=default.Table_786f03e0f3404d69a9f8003f5e62368c [id=a6476fbd5ce8492e8d4e54238c5c019f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.288261 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b4e886bebc4ab6a3c95290d18b4229. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.291814 749 tablet_service.cc:1511] Processing CreateTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 (DEFAULT_TABLE table=default.Table_2717154d3561495692818d7755d53c41 [id=23c6264859f64b598af609ed2fff6e8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.292148 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba616702dc6040bb805bb3ffc8a8a871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.295101 1176 tablet_bootstrap.cc:654] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.296957 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 09a8dbb7fe4b406d804c0b62ba006adb (table default.Table_eae91d411d9149658200d44f4837f486 [id=789cc9783535475e9929332f36141e27]) successfully deleted
I20260505 07:53:58.297654 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 5ad8a79245ad47738d079752e51aed2b
I20260505 07:53:58.298049 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 1602ccbae24544c4a6cf14ea073880b4
I20260505 07:53:58.298873 897 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ad8a79245ad47738d079752e51aed2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.299088 886 tablet_service.cc:1558] Processing DeleteTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.299319 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ad8a79245ad47738d079752e51aed2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.300117 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.301273 886 tablet_service.cc:1511] Processing CreateTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 (DEFAULT_TABLE table=default.Table_d4d6e315fb8340e9b629d876d050ef05 [id=5fdbc0ad9c3f4e0a9b2fc0997a87870b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.302114 897 tablet_service.cc:1511] Processing CreateTablet for tablet 54e4f7f895d943639653c8c32724baf2 (DEFAULT_TABLE table=default.Table_1b6fae561a60438aa62f5edf8eaea6c4 [id=54621fbfaf6a4b4a874207bd986b77be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.302206 750 tablet_service.cc:1511] Processing CreateTablet for tablet 54e4f7f895d943639653c8c32724baf2 (DEFAULT_TABLE table=default.Table_1b6fae561a60438aa62f5edf8eaea6c4 [id=54621fbfaf6a4b4a874207bd986b77be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.302544 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54e4f7f895d943639653c8c32724baf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.304203 1175 tablet_replica.cc:333] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:58.304905 1175 raft_consensus.cc:2243] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.305028 754 tablet_service.cc:1511] Processing CreateTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 (DEFAULT_TABLE table=default.Table_d4d6e315fb8340e9b629d876d050ef05 [id=5fdbc0ad9c3f4e0a9b2fc0997a87870b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.305994 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_82c755fca0b64337a44a63876e04fa68" } modify_external_catalogs: true
I20260505 07:53:58.306154 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_82c755fca0b64337a44a63876e04fa68" } modify_external_catalogs: true
I20260505 07:53:58.309568 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 54e4f7f895d943639653c8c32724baf2 (DEFAULT_TABLE table=default.Table_1b6fae561a60438aa62f5edf8eaea6c4 [id=54621fbfaf6a4b4a874207bd986b77be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.309962 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 17b4e886bebc4ab6a3c95290d18b4229 (DEFAULT_TABLE table=default.Table_786f03e0f3404d69a9f8003f5e62368c [id=a6476fbd5ce8492e8d4e54238c5c019f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.310132 883 tablet_service.cc:1511] Processing CreateTablet for tablet f200abef13ac42698a9b96fc8af82ff0 (DEFAULT_TABLE table=default.Table_157815480b75424885451ec833e3438d [id=ebd83e4e101d4282ab8aeec21c281c60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.310781 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54e4f7f895d943639653c8c32724baf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.311461 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b406c4c6ba741bc85eef54c6f576459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.311631 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b4e886bebc4ab6a3c95290d18b4229. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.311914 761 tablet_service.cc:1511] Processing CreateTablet for tablet f200abef13ac42698a9b96fc8af82ff0 (DEFAULT_TABLE table=default.Table_157815480b75424885451ec833e3438d [id=ebd83e4e101d4282ab8aeec21c281c60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.314352 882 tablet_service.cc:1511] Processing CreateTablet for tablet 76939b7978ee4f7ea0ad29312de2a31a (DEFAULT_TABLE table=default.Table_cf18ef8e6a56482883dba2744997604a [id=b131fb4265684cd2b6506a7abeb29871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.315662 1178 tablet_replica.cc:333] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:58.316128 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 26f3544e74d941d286a2a0317b941ab6
I20260505 07:53:58.317147 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 26f3544e74d941d286a2a0317b941ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.317157 1020 tablet_service.cc:1511] Processing CreateTablet for tablet 76939b7978ee4f7ea0ad29312de2a31a (DEFAULT_TABLE table=default.Table_cf18ef8e6a56482883dba2744997604a [id=b131fb4265684cd2b6506a7abeb29871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.318163 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 26f3544e74d941d286a2a0317b941ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.318542 890 tablet_service.cc:1511] Processing CreateTablet for tablet df1b62a30b2842b897a04c793bb5e7ce (DEFAULT_TABLE table=default.Table_b64f571bfd004207bd2fb763c53b9502 [id=269f2d2935a644518501cb0700a7cff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.319108 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_e57994d74efd45c488c52378e0548b36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:58.319227 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1b6fae561a60438aa62f5edf8eaea6c4" } new_table_name: "default.Table_433dccceaf294d6facc01e2ea5297a9f" modify_external_catalogs: true
I20260505 07:53:58.319182 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ff8ae2444af4400eb86af8cc0fa3c146"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.319664 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e57994d74efd45c488c52378e0548b36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:58.319732 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff8ae2444af4400eb86af8cc0fa3c146 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.321033 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f8f7a899e8c94e75958bd11fed643bc6" } modify_external_catalogs: true
I20260505 07:53:58.321264 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f8f7a899e8c94e75958bd11fed643bc6" } modify_external_catalogs: true
I20260505 07:53:58.323292 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ad8a79245ad47738d079752e51aed2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:58.323812 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 840eebf9e7404bf7807b7e51996957e5 consensus state has no leader
I20260505 07:53:58.326790 893 tablet_service.cc:1511] Processing CreateTablet for tablet 6fe19306f7df49fdb771d21f6560e697 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.326742 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.328123 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f200abef13ac42698a9b96fc8af82ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.327855 1027 tablet_service.cc:1511] Processing CreateTablet for tablet df1b62a30b2842b897a04c793bb5e7ce (DEFAULT_TABLE table=default.Table_b64f571bfd004207bd2fb763c53b9502 [id=269f2d2935a644518501cb0700a7cff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.328974 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 6fe19306f7df49fdb771d21f6560e697 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.330217 1034 tablet_service.cc:1511] Processing CreateTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 (DEFAULT_TABLE table=default.Table_2717154d3561495692818d7755d53c41 [id=23c6264859f64b598af609ed2fff6e8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.331156 1032 tablet_service.cc:1511] Processing CreateTablet for tablet f200abef13ac42698a9b96fc8af82ff0 (DEFAULT_TABLE table=default.Table_157815480b75424885451ec833e3438d [id=ebd83e4e101d4282ab8aeec21c281c60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.331564 1026 tablet_service.cc:1511] Processing CreateTablet for tablet e1a403e4708e4725899c1cd5b9fb0e8b (DEFAULT_TABLE table=default.Table_8564f8e67e6f4995b312673dd0d63f52 [id=de1506e299994d1fbe31033d301d4533]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.332986 746 tablet_service.cc:1511] Processing CreateTablet for tablet 76939b7978ee4f7ea0ad29312de2a31a (DEFAULT_TABLE table=default.Table_cf18ef8e6a56482883dba2744997604a [id=b131fb4265684cd2b6506a7abeb29871]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.333868 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76939b7978ee4f7ea0ad29312de2a31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.335404 755 tablet_service.cc:1511] Processing CreateTablet for tablet 6fe19306f7df49fdb771d21f6560e697 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.335443 887 tablet_service.cc:1511] Processing CreateTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 (DEFAULT_TABLE table=default.Table_73ba90aede024514abb230661a44a4ae [id=03a241143fae44a19531301524ef4adb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.335776 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fe19306f7df49fdb771d21f6560e697. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:58.335899 991 consensus_peers.cc:597] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8cfe212cb6c14ce3a63b6ee4275f66f1. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:53:58.336620 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 26f3544e74d941d286a2a0317b941ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.336961 744 tablet_service.cc:1511] Processing CreateTablet for tablet df1b62a30b2842b897a04c793bb5e7ce (DEFAULT_TABLE table=default.Table_b64f571bfd004207bd2fb763c53b9502 [id=269f2d2935a644518501cb0700a7cff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.337359 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df1b62a30b2842b897a04c793bb5e7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.339418 1175 raft_consensus.cc:2272] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.340265 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a403e4708e4725899c1cd5b9fb0e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.340492 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76939b7978ee4f7ea0ad29312de2a31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.340730 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df1b62a30b2842b897a04c793bb5e7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.340939 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b406c4c6ba741bc85eef54c6f576459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.341178 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0db162dfad2448e8ac428b3d3fbe17f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.341388 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f200abef13ac42698a9b96fc8af82ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.341656 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fe19306f7df49fdb771d21f6560e697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.341894 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54e4f7f895d943639653c8c32724baf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.348762 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17b4e886bebc4ab6a3c95290d18b4229. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.352010 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba616702dc6040bb805bb3ffc8a8a871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.352596 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_c1551c1a06c34c5dae05b3d5f1109d43"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.353197 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1551c1a06c34c5dae05b3d5f1109d43 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.353873 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6121e74274264813a8cc8dd21d0d5e17" } modify_external_catalogs: true
I20260505 07:53:58.354075 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6121e74274264813a8cc8dd21d0d5e17" } modify_external_catalogs: true
I20260505 07:53:58.354104 599 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f7876fb512074cf88d88f99db4ae1720" } new_table_name: "default.Table_d8d1ac5a03af445490dd8c36db28f4be" modify_external_catalogs: true
I20260505 07:53:58.355612 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_45faf5ec25414b7e8126ba77d62ca031" } new_table_name: "default.Table_5613a0267c774633aabe68f172339eea" modify_external_catalogs: true
I20260505 07:53:58.359462 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6cb216f92da74a78acd72a6132b94c1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.360090 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6cb216f92da74a78acd72a6132b94c1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.361186 751 tablet_service.cc:1511] Processing CreateTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 (DEFAULT_TABLE table=default.Table_73ba90aede024514abb230661a44a4ae [id=03a241143fae44a19531301524ef4adb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.361642 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0db162dfad2448e8ac428b3d3fbe17f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.341223 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 (DEFAULT_TABLE table=default.Table_d4d6e315fb8340e9b629d876d050ef05 [id=5fdbc0ad9c3f4e0a9b2fc0997a87870b]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:58.368388 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 54e4f7f895d943639653c8c32724baf2 consensus state has no leader
I20260505 07:53:58.369704 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet a0f2f46d0e7840458cb992ce2a2dbea2
W20260505 07:53:58.375468 599 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 840eebf9e7404bf7807b7e51996957e5 consensus state has no leader
W20260505 07:53:58.375517 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5683d7366de84653ace56f0edcd3818c consensus state has no leader
I20260505 07:53:58.376040 595 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 4b1b58a598e5478f89f73fa21897f412
I20260505 07:53:58.377509 1175 ts_tablet_manager.cc:1916] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.386917 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 (DEFAULT_TABLE table=default.Table_73ba90aede024514abb230661a44a4ae [id=03a241143fae44a19531301524ef4adb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.388618 1171 ts_tablet_manager.cc:1916] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.388870 752 tablet_service.cc:1511] Processing CreateTablet for tablet a1175f7deec14fe39098ec89ab3bc049 (DEFAULT_TABLE table=default.Table_ed3f6be0222a4436802966dd49d1dfad [id=bccdbf7ecfdd40c59dc52c4562f88728]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.389235 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1175f7deec14fe39098ec89ab3bc049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.389429 887 tablet_service.cc:1558] Processing DeleteTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.390583 758 tablet_service.cc:1558] Processing DeleteTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.401145 887 tablet_service.cc:1511] Processing CreateTablet for tablet a1175f7deec14fe39098ec89ab3bc049 (DEFAULT_TABLE table=default.Table_ed3f6be0222a4436802966dd49d1dfad [id=bccdbf7ecfdd40c59dc52c4562f88728]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.406262 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_c1cf7717be07400aacdab563b4255450"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:58.409968 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1175f7deec14fe39098ec89ab3bc049. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:58.413341 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c1cf7717be07400aacdab563b4255450 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.421675 597 master_service.cc:954] ReplaceTablet: received request to replace tablet c1a61c1d03d544c8aa01537ead97f83c from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:58.422227 898 tablet_service.cc:1511] Processing CreateTablet for tablet f84f464e83cb412eade9978ea99f6685 (DEFAULT_TABLE table=default.Table_d023acedd7c842ebb396d7354c41b490 [id=a91d46106bbd4967ad97e5de33b59891]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.422259 896 tablet_service.cc:1511] Processing CreateTablet for tablet 66082a7a75e345639e7a268727922fbb (DEFAULT_TABLE table=default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.424265 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f84f464e83cb412eade9978ea99f6685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.425609 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66082a7a75e345639e7a268727922fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.432798 749 tablet_service.cc:1511] Processing CreateTablet for tablet f84f464e83cb412eade9978ea99f6685 (DEFAULT_TABLE table=default.Table_d023acedd7c842ebb396d7354c41b490 [id=a91d46106bbd4967ad97e5de33b59891]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.429032 758 tablet_service.cc:1511] Processing CreateTablet for tablet 472441bd29704be59aabb0f7a4964bb0 (DEFAULT_TABLE table=default.Table_8ef2c24cce2148ecb1409ecaa29a8130 [id=c82e4e1fd6004306906c95957b86620f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.434103 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f84f464e83cb412eade9978ea99f6685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.437825 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 472441bd29704be59aabb0f7a4964bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.438561 757 tablet_service.cc:1511] Processing CreateTablet for tablet e010f23c4df647ab863e03c3cfed7898 (DEFAULT_TABLE table=default.Table_55630f6d78594e3d810e3291317b348c [id=09bd9815349347ab9b75c4242c354c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.438963 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e010f23c4df647ab863e03c3cfed7898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.440068 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet c1a61c1d03d544c8aa01537ead97f83c
I20260505 07:53:58.440589 597 catalog_manager.cc:6741] ReplaceTablet: tablet c1a61c1d03d544c8aa01537ead97f83c deleted and replaced by tablet 080b2984fcf2478a8def6540da63554d
I20260505 07:53:58.447330 901 tablet_service.cc:1511] Processing CreateTablet for tablet e010f23c4df647ab863e03c3cfed7898 (DEFAULT_TABLE table=default.Table_55630f6d78594e3d810e3291317b348c [id=09bd9815349347ab9b75c4242c354c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.447538 760 tablet_service.cc:1511] Processing CreateTablet for tablet 4e99f96a753047348371bee5c022684e (DEFAULT_TABLE table=default.Table_55f3a1ca8c53455993b486b0798f0425 [id=fb5d62bb12c5499e8953cb3b431867a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.447790 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e010f23c4df647ab863e03c3cfed7898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.449635 895 tablet_service.cc:1511] Processing CreateTablet for tablet 4e99f96a753047348371bee5c022684e (DEFAULT_TABLE table=default.Table_55f3a1ca8c53455993b486b0798f0425 [id=fb5d62bb12c5499e8953cb3b431867a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.449891 756 tablet_service.cc:1558] Processing DeleteTablet for tablet c1a61c1d03d544c8aa01537ead97f83c with delete_type TABLET_DATA_DELETED (replaced by tablet 080b2984fcf2478a8def6540da63554d) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.450043 1176 tablet_bootstrap.cc:492] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:58.450819 1176 ts_tablet_manager.cc:1403] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.269s user 0.007s sys 0.004s
I20260505 07:53:58.453001 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e99f96a753047348371bee5c022684e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.484714 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_119772d89d5f4afa8108d78252d81fe6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.485576 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_119772d89d5f4afa8108d78252d81fe6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.485857 599 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_55630f6d78594e3d810e3291317b348c" } new_table_name: "default.Table_866ca8833a3a4cb6a04c41d3b7b76c23" modify_external_catalogs: true
I20260505 07:53:58.486047 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_fa7a8dd4d4e8453180804e6aa56d8ca4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.486497 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fa7a8dd4d4e8453180804e6aa56d8ca4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.487243 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_0bda6aa949274f898514caa0a79767d3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.487835 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0bda6aa949274f898514caa0a79767d3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.488482 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_13e8ecc0f18d4b909d161b6b56661449"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.489245 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_13e8ecc0f18d4b909d161b6b56661449 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.484802 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d2caf74ce2d94a2ea94685efc828218b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.490075 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2caf74ce2d94a2ea94685efc828218b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.500237 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ff8ae2444af4400eb86af8cc0fa3c146" } modify_external_catalogs: true
I20260505 07:53:58.500447 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ff8ae2444af4400eb86af8cc0fa3c146" } modify_external_catalogs: true
I20260505 07:53:58.500937 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_66ccaae918c545228101f7d0e29c37dc" } modify_external_catalogs: true
I20260505 07:53:58.501072 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_66ccaae918c545228101f7d0e29c37dc" } modify_external_catalogs: true
I20260505 07:53:58.451325 884 tablet_service.cc:1558] Processing DeleteTablet for tablet c1a61c1d03d544c8aa01537ead97f83c with delete_type TABLET_DATA_DELETED (replaced by tablet 080b2984fcf2478a8def6540da63554d) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.455308 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e99f96a753047348371bee5c022684e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.496160 1171 ts_tablet_manager.cc:1929] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.513350 1176 raft_consensus.cc:359] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.514093 1176 raft_consensus.cc:385] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.450839 893 tablet_service.cc:1511] Processing CreateTablet for tablet 472441bd29704be59aabb0f7a4964bb0 (DEFAULT_TABLE table=default.Table_8ef2c24cce2148ecb1409ecaa29a8130 [id=c82e4e1fd6004306906c95957b86620f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.514264 1176 raft_consensus.cc:740] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.514966 1176 consensus_queue.cc:260] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.520367 762 tablet_service.cc:1511] Processing CreateTablet for tablet 66082a7a75e345639e7a268727922fbb (DEFAULT_TABLE table=default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.523452 1175 ts_tablet_manager.cc:1929] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.525010 1171 log.cc:1199] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/4a79578757824bfc9df129e3475e7af3
I20260505 07:53:58.525767 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_157815480b75424885451ec833e3438d" } modify_external_catalogs: true
I20260505 07:53:58.525964 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_157815480b75424885451ec833e3438d" } modify_external_catalogs: true
I20260505 07:53:58.528012 593 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d27b912f6fe14120afb18354e772a658" } new_table_name: "default.Table_2872bda04bf547648372a8f4bf5dc3ce" modify_external_catalogs: true
I20260505 07:53:58.529589 1175 log.cc:1199] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/ce9d01b0b5464a65966f26839b48aaf1
I20260505 07:53:58.529981 1175 ts_tablet_manager.cc:1950] T ce9d01b0b5464a65966f26839b48aaf1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:58.530972 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet ce9d01b0b5464a65966f26839b48aaf1 (table default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]) successfully deleted
W20260505 07:53:58.531725 599 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e010f23c4df647ab863e03c3cfed7898 consensus state has no leader
W20260505 07:53:58.531893 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet bf76e868bbcb44b0b1887c84dd27f093 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf76e868bbcb44b0b1887c84dd27f093 already in progress: creating tablet
I20260505 07:53:58.532352 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66082a7a75e345639e7a268727922fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.539644 1171 ts_tablet_manager.cc:1950] T 4a79578757824bfc9df129e3475e7af3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:58.540676 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 4a79578757824bfc9df129e3475e7af3 (table default.Table_b31d40ee188f46d0bdd481aa88730544 [id=c6653c0d02c64683870dc0e9551fcaf5]) successfully deleted
W20260505 07:53:58.542280 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 5ad8a79245ad47738d079752e51aed2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ad8a79245ad47738d079752e51aed2b already in progress: creating tablet
W20260505 07:53:58.543147 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 1602ccbae24544c4a6cf14ea073880b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1602ccbae24544c4a6cf14ea073880b4 already in progress: creating tablet
W20260505 07:53:58.544035 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 26f3544e74d941d286a2a0317b941ab6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26f3544e74d941d286a2a0317b941ab6 already in progress: creating tablet
W20260505 07:53:58.544960 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a0f2f46d0e7840458cb992ce2a2dbea2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0f2f46d0e7840458cb992ce2a2dbea2 already in progress: creating tablet
W20260505 07:53:58.545850 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet c1a61c1d03d544c8aa01537ead97f83c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1a61c1d03d544c8aa01537ead97f83c already in progress: creating tablet
I20260505 07:53:58.550647 1176 ts_tablet_manager.cc:1434] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.091s user 0.001s sys 0.003s
I20260505 07:53:58.551712 1176 tablet_bootstrap.cc:492] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:58.551749 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 472441bd29704be59aabb0f7a4964bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.554378 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 1765a95cc3414c73b7e3918913a35937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.554968 1178 raft_consensus.cc:2243] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.559008 1172 tablet_bootstrap.cc:492] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:58.560189 1172 ts_tablet_manager.cc:1403] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.327s user 0.005s sys 0.005s
I20260505 07:53:58.561169 1172 raft_consensus.cc:359] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:58.562920 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 5ad8a79245ad47738d079752e51aed2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ad8a79245ad47738d079752e51aed2b already in progress: creating tablet
I20260505 07:53:58.570398 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b7306feea82e49bc917b31d7feb4f1e0" } new_table_name: "default.Table_44ead87f92584e6e98e2c5f69056591a" modify_external_catalogs: true
W20260505 07:53:58.574038 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 1602ccbae24544c4a6cf14ea073880b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1602ccbae24544c4a6cf14ea073880b4 already in progress: creating tablet
I20260505 07:53:58.579540 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 76939b7978ee4f7ea0ad29312de2a31a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.579942 1035 tablet_service.cc:1511] Processing CreateTablet for tablet a1175f7deec14fe39098ec89ab3bc049 (DEFAULT_TABLE table=default.Table_ed3f6be0222a4436802966dd49d1dfad [id=bccdbf7ecfdd40c59dc52c4562f88728]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.581745 1178 raft_consensus.cc:2272] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:53:58.585219 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 26f3544e74d941d286a2a0317b941ab6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26f3544e74d941d286a2a0317b941ab6 already in progress: creating tablet
I20260505 07:53:58.587702 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_28c935edd66e47959a2df29d39dfe13c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.588261 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_28c935edd66e47959a2df29d39dfe13c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:58.589265 988 consensus_peers.cc:597] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8cfe212cb6c14ce3a63b6ee4275f66f1. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:53:58.589375 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 154905us
I20260505 07:53:58.590037 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df1b62a30b2842b897a04c793bb5e7ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.590912 1029 tablet_service.cc:1558] Processing DeleteTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.593919 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2717154d3561495692818d7755d53c41" } modify_external_catalogs: true
I20260505 07:53:58.594100 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2717154d3561495692818d7755d53c41" } modify_external_catalogs: true
I20260505 07:53:58.595137 1022 tablet_service.cc:1511] Processing CreateTablet for tablet f84f464e83cb412eade9978ea99f6685 (DEFAULT_TABLE table=default.Table_d023acedd7c842ebb396d7354c41b490 [id=a91d46106bbd4967ad97e5de33b59891]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.595512 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 472441bd29704be59aabb0f7a4964bb0 (DEFAULT_TABLE table=default.Table_8ef2c24cce2148ecb1409ecaa29a8130 [id=c82e4e1fd6004306906c95957b86620f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:58.596882 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet a0f2f46d0e7840458cb992ce2a2dbea2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0f2f46d0e7840458cb992ce2a2dbea2 already in progress: creating tablet
I20260505 07:53:58.604076 1016 tablet_service.cc:1558] Processing DeleteTablet for tablet c1a61c1d03d544c8aa01537ead97f83c with delete_type TABLET_DATA_DELETED (replaced by tablet 080b2984fcf2478a8def6540da63554d) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.604219 1021 tablet_service.cc:1511] Processing CreateTablet for tablet e010f23c4df647ab863e03c3cfed7898 (DEFAULT_TABLE table=default.Table_55630f6d78594e3d810e3291317b348c [id=09bd9815349347ab9b75c4242c354c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.604528 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 66082a7a75e345639e7a268727922fbb (DEFAULT_TABLE table=default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.604815 1178 ts_tablet_manager.cc:1916] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:58.610903 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet c1a61c1d03d544c8aa01537ead97f83c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c1a61c1d03d544c8aa01537ead97f83c already in progress: creating tablet
I20260505 07:53:58.615031 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba616702dc6040bb805bb3ffc8a8a871. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.615746 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 4e99f96a753047348371bee5c022684e (DEFAULT_TABLE table=default.Table_55f3a1ca8c53455993b486b0798f0425 [id=fb5d62bb12c5499e8953cb3b431867a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.621794 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e010f23c4df647ab863e03c3cfed7898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.622130 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e99f96a753047348371bee5c022684e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.627713 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0db162dfad2448e8ac428b3d3fbe17f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.631225 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1a403e4708e4725899c1cd5b9fb0e8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.631753 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fe19306f7df49fdb771d21f6560e697. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.631956 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 472441bd29704be59aabb0f7a4964bb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.634162 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6f171a1e79d24e57acfe3aef43b24c7e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.634684 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6f171a1e79d24e57acfe3aef43b24c7e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.635844 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b406c4c6ba741bc85eef54c6f576459. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.636068 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f84f464e83cb412eade9978ea99f6685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.639493 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66082a7a75e345639e7a268727922fbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.640092 1175 tablet_replica.cc:333] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:58.642539 1172 raft_consensus.cc:385] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.647627 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1175f7deec14fe39098ec89ab3bc049. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.651394 601 master_service.cc:954] ReplaceTablet: received request to replace tablet 3d6494590052459485059f5dce46f34f from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:58.653743 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_e57994d74efd45c488c52378e0548b36" } new_table_name: "default.Table_314825f5a7f644dba828c34f3d2d740a" modify_external_catalogs: true
I20260505 07:53:58.663630 1172 raft_consensus.cc:740] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.665174 1172 consensus_queue.cc:260] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.665589 1178 ts_tablet_manager.cc:1929] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.665781 1178 log.cc:1199] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/2850a301ac4f435a96742b00bf54baa9
I20260505 07:53:58.665858 1172 ts_tablet_manager.cc:1434] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.106s user 0.001s sys 0.001s
I20260505 07:53:58.660382 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6b95372e816b4dac9493d85379e26080" } modify_external_catalogs: true
I20260505 07:53:58.666244 1178 ts_tablet_manager.cc:1950] T 2850a301ac4f435a96742b00bf54baa9 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:58.666261 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6b95372e816b4dac9493d85379e26080" } modify_external_catalogs: true
I20260505 07:53:58.670275 1172 tablet_bootstrap.cc:492] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:58.672242 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet f200abef13ac42698a9b96fc8af82ff0
I20260505 07:53:58.672559 1172 tablet_bootstrap.cc:654] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.677289 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7eb9890f5e7349748b35f1134174580e" } new_table_name: "default.Table_ad3e5b8653224a0c929e29e9daf47ff5" modify_external_catalogs: true
I20260505 07:53:58.678668 901 tablet_service.cc:1558] Processing DeleteTablet for tablet f200abef13ac42698a9b96fc8af82ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.672595 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f200abef13ac42698a9b96fc8af82ff0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:58.680244 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet f200abef13ac42698a9b96fc8af82ff0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f200abef13ac42698a9b96fc8af82ff0 already in progress: creating tablet
W20260505 07:53:58.684063 593 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f1058fcb9fa6457a85489049ca1d9ada consensus state has no leader
I20260505 07:53:58.677124 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet f200abef13ac42698a9b96fc8af82ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.693868 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ed8e2add9bbf436f812de7e3fe2ed692"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.696033 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed8e2add9bbf436f812de7e3fe2ed692 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.696202 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 03f31efbc98c484c950c70d5e84d2136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.696539 1171 tablet_replica.cc:333] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:58.696767 1171 raft_consensus.cc:2243] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.696933 1171 raft_consensus.cc:2272] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.697391 1175 raft_consensus.cc:2243] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.697893 1171 ts_tablet_manager.cc:1916] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:58.702194 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 297537a92684425d8508f7b3b4c9d379 consensus state has no leader
I20260505 07:53:58.704025 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ca13a11f3fd249eabadc7213e7318acd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.704633 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca13a11f3fd249eabadc7213e7318acd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.706771 1176 tablet_bootstrap.cc:654] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.708895 757 tablet_service.cc:1558] Processing DeleteTablet for tablet f200abef13ac42698a9b96fc8af82ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.709312 1175 raft_consensus.cc:2272] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.724028 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 232) took 1499 ms. Trace:
I20260505 07:53:58.724141 1178 rpcz_store.cc:276] 0505 07:53:57.224044 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.226052 (+ 2008us) service_pool.cc:224] Handling call
0505 07:53:58.301022 (+1074970us) ts_tablet_manager.cc:1151] Deleting tablet 2850a301ac4f435a96742b00bf54baa9
0505 07:53:58.301030 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:58.647231 (+346201us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.665515 (+ 18284us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.723796 (+ 58281us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1070909}
I20260505 07:53:58.724589 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 2850a301ac4f435a96742b00bf54baa9 (table default.Table_017976a3a6af42c9a9fc722661286c0e [id=d524458d873a48cb980e2f85490232d6]) successfully deleted
I20260505 07:53:58.725051 1178 tablet_replica.cc:333] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:58.725234 1178 raft_consensus.cc:2243] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.725782 1178 raft_consensus.cc:2272] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.726899 1178 ts_tablet_manager.cc:1916] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.728909 1172 tablet_bootstrap.cc:492] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:58.729058 1172 ts_tablet_manager.cc:1403] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.059s user 0.003s sys 0.000s
W20260505 07:53:58.730039 1181 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.515s user 0.032s sys 0.089s
W20260505 07:53:58.730132 1181 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.515s user 0.032s sys 0.089s
I20260505 07:53:58.730091 1172 raft_consensus.cc:359] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.730211 1171 ts_tablet_manager.cc:1929] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.730315 1172 raft_consensus.cc:385] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.730921 1172 raft_consensus.cc:740] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.733580 1175 ts_tablet_manager.cc:1916] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.736994 1181 tablet_bootstrap.cc:492] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:58.737236 1181 ts_tablet_manager.cc:1403] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.629s user 0.034s sys 0.092s
I20260505 07:53:58.738876 1181 raft_consensus.cc:359] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.739223 1181 raft_consensus.cc:385] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.739553 1181 raft_consensus.cc:740] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.750425 1172 consensus_queue.cc:260] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.751300 1172 ts_tablet_manager.cc:1434] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.022s user 0.002s sys 0.001s
I20260505 07:53:58.750788 1171 log.cc:1199] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/03f31efbc98c484c950c70d5e84d2136
I20260505 07:53:58.751909 1171 ts_tablet_manager.cc:1950] T 03f31efbc98c484c950c70d5e84d2136 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:58.752460 1172 tablet_bootstrap.cc:492] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:58.754000 1181 consensus_queue.cc:260] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.754252 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_359f92a4ff554e3e80468261b0e86966"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.756250 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_359f92a4ff554e3e80468261b0e86966 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.774358 1181 ts_tablet_manager.cc:1434] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.037s user 0.002s sys 0.000s
I20260505 07:53:58.776876 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet ba616702dc6040bb805bb3ffc8a8a871
I20260505 07:53:58.781910 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.782644 1181 tablet_bootstrap.cc:492] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:58.782915 758 tablet_service.cc:1558] Processing DeleteTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.784938 1181 tablet_bootstrap.cc:654] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.786576 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 03f31efbc98c484c950c70d5e84d2136 (table default.Table_6ff000247d324fe8b55b1398e48cb49a [id=8903351259c347a1925769663d34d173]) successfully deleted
I20260505 07:53:58.788470 598 master_service.cc:954] ReplaceTablet: received request to replace tablet 8ca4cf248adb4f768c5586d72614ad42 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:58.790261 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 8573f8bd424641438e2aa6ff6869180e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.781369 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 9959be9f83c2444282d012b61eef49b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.792264 600 master_service.cc:954] ReplaceTablet: received request to replace tablet a0f2f46d0e7840458cb992ce2a2dbea2 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:58.784081 893 tablet_service.cc:1558] Processing DeleteTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.793314 1172 tablet_bootstrap.cc:654] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.794656 1171 tablet_replica.cc:333] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:58.794849 1171 raft_consensus.cc:2243] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.795053 1171 raft_consensus.cc:2272] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.795810 1171 ts_tablet_manager.cc:1916] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.796842 600 master_service.cc:954] ReplaceTablet: received request to replace tablet 1838f89dc14a47f99f95f4cc5fdefd70 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:58.797331 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_fd8baf98a0f64d7fbd3d61d32af60991" } new_table_name: "default.Table_4bd0f10670d240ee8aa468cf48521647" modify_external_catalogs: true
I20260505 07:53:58.799618 1178 ts_tablet_manager.cc:1929] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.799808 1178 log.cc:1199] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/62d4d96bdf674c2395938b9971c0ad28
I20260505 07:53:58.800246 1178 ts_tablet_manager.cc:1950] T 62d4d96bdf674c2395938b9971c0ad28 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:58.800518 1176 tablet_bootstrap.cc:492] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:58.800704 1176 ts_tablet_manager.cc:1403] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.249s user 0.007s sys 0.004s
I20260505 07:53:58.801182 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 237) took 1561 ms. Trace:
I20260505 07:53:58.801345 1178 rpcz_store.cc:276] 0505 07:53:57.240072 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.240268 (+ 196us) service_pool.cc:224] Handling call
0505 07:53:58.724850 (+1484582us) ts_tablet_manager.cc:1151] Deleting tablet 62d4d96bdf674c2395938b9971c0ad28
0505 07:53:58.724860 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:58.749329 (+ 24469us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.799486 (+ 50157us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.801146 (+ 1660us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1484398}
I20260505 07:53:58.801558 1176 raft_consensus.cc:359] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.801748 1176 raft_consensus.cc:385] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.801851 1176 raft_consensus.cc:740] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.802032 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 62d4d96bdf674c2395938b9971c0ad28 (table default.Table_f0316c88590245aea8815b3b08e89f45 [id=c3a480b747de444aa45b771244908352]) successfully deleted
I20260505 07:53:58.802119 1176 consensus_queue.cc:260] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.802672 1176 ts_tablet_manager.cc:1434] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260505 07:53:58.806352 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d7e0c8ad716e443588d4ae3aa28ed4bf has no consensus state
I20260505 07:53:58.808300 1181 tablet_bootstrap.cc:492] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:58.808529 1181 ts_tablet_manager.cc:1403] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
I20260505 07:53:58.809545 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d4c7087457154e89b20abe713355e422"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:58.809558 1181 raft_consensus.cc:359] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.809860 1181 raft_consensus.cc:385] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.809952 1181 raft_consensus.cc:740] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
W20260505 07:53:58.810199 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d4c7087457154e89b20abe713355e422 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.810302 1178 tablet_replica.cc:333] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:58.810240 1181 consensus_queue.cc:260] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.817226 1175 ts_tablet_manager.cc:1929] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.818295 1172 tablet_bootstrap.cc:492] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:58.818677 1172 ts_tablet_manager.cc:1403] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.066s user 0.002s sys 0.002s
I20260505 07:53:58.819525 1172 raft_consensus.cc:359] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:58.820029 1172 raft_consensus.cc:385] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.820111 1172 raft_consensus.cc:740] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.820361 1172 consensus_queue.cc:260] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:58.820703 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 3781d0bbcd1b4189bd836eeae4236f40
I20260505 07:53:58.820835 1172 ts_tablet_manager.cc:1434] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:53:58.821379 1178 raft_consensus.cc:2243] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.821676 1172 tablet_bootstrap.cc:492] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:58.823619 893 tablet_service.cc:1558] Processing DeleteTablet for tablet 3781d0bbcd1b4189bd836eeae4236f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.825537 1172 tablet_bootstrap.cc:654] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.827467 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 3781d0bbcd1b4189bd836eeae4236f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.827729 1181 ts_tablet_manager.cc:1434] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.019s user 0.001s sys 0.002s
I20260505 07:53:58.828963 1181 tablet_bootstrap.cc:492] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:58.831517 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 451e99a0dfec4823812e5991147b132e with delete_type TABLET_DATA_DELETED (replaced by tablet fcd36b5a84b4442fb76e1610d8af4c07) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.828169 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 3781d0bbcd1b4189bd836eeae4236f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.831897 1175 log.cc:1199] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/1765a95cc3414c73b7e3918913a35937
I20260505 07:53:58.832350 1175 ts_tablet_manager.cc:1950] T 1765a95cc3414c73b7e3918913a35937 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:58.833151 1178 raft_consensus.cc:2272] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:53:58.835783 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet feeafd614b3e4281a8e45b950e699b63 consensus state has no leader
I20260505 07:53:58.836637 1181 tablet_bootstrap.cc:654] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.839121 1178 ts_tablet_manager.cc:1916] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.828281 1171 ts_tablet_manager.cc:1929] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.840443 592 master_service.cc:954] ReplaceTablet: received request to replace tablet 9959be9f83c2444282d012b61eef49b5 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:58.840327 1171 log.cc:1199] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/9959be9f83c2444282d012b61eef49b5
I20260505 07:53:58.841015 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_657cc0ba6f494479bd433c8c0279c734"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:58.841277 1171 ts_tablet_manager.cc:1950] T 9959be9f83c2444282d012b61eef49b5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
W20260505 07:53:58.841576 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_657cc0ba6f494479bd433c8c0279c734 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.841912 1490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d3eda04f9c47ae976d19c218b6dcc5. Latest consensus state: current_term: 1 leader_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:58.842121 1490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:53:58.842286 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 9959be9f83c2444282d012b61eef49b5 (table default.Table_ae34ec4a8b52453dba7158ce065f4798 [id=8314d2bc7c1a401db1d6fac29fff5c3f]) successfully deleted
W20260505 07:53:58.842926 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet ba616702dc6040bb805bb3ffc8a8a871 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba616702dc6040bb805bb3ffc8a8a871 already in progress: creating tablet
I20260505 07:53:58.843106 1518 raft_consensus.cc:493] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28d3eda04f9c47ae976d19c218b6dcc5)
I20260505 07:53:58.843442 1171 tablet_replica.cc:333] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:58.843377 1518 raft_consensus.cc:515] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:53:58.844264 1171 raft_consensus.cc:2243] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.848084 1176 tablet_bootstrap.cc:492] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:53:58.848780 586 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet f1058fcb9fa6457a85489049ca1d9ada (table default.Table_2872bda04bf547648372a8f4bf5dc3ce [id=42ae40c0861e4489b724c9b8e3b590d7], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f1058fcb9fa6457a85489049ca1d9ada consensus state has no leader
I20260505 07:53:58.856724 1181 tablet_bootstrap.cc:492] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:58.856914 1181 ts_tablet_manager.cc:1403] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.029s user 0.002s sys 0.002s
I20260505 07:53:58.858040 1171 raft_consensus.cc:2272] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.858417 1181 raft_consensus.cc:359] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.858469 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_79ed240074b343fabbdd206872328277"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:58.858671 1181 raft_consensus.cc:385] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.858776 1181 raft_consensus.cc:740] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
W20260505 07:53:58.859053 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79ed240074b343fabbdd206872328277 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.859081 1181 consensus_queue.cc:260] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.860651 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 8573f8bd424641438e2aa6ff6869180e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.862717 1181 ts_tablet_manager.cc:1434] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260505 07:53:58.863075 1181 tablet_bootstrap.cc:492] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:58.864018 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 1765a95cc3414c73b7e3918913a35937 (table default.Table_a006de2dda30423f83e51da1e0477f9d [id=13ca7143dd0249d597c79bbfc17ae7f9]) successfully deleted
I20260505 07:53:58.864274 1175 tablet_replica.cc:333] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:58.864496 1175 raft_consensus.cc:2243] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260505 07:53:58.864528 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet f200abef13ac42698a9b96fc8af82ff0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f200abef13ac42698a9b96fc8af82ff0 already in progress: creating tablet
I20260505 07:53:58.864698 1175 raft_consensus.cc:2272] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.865134 1181 tablet_bootstrap.cc:654] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:58.865296 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet ba616702dc6040bb805bb3ffc8a8a871 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba616702dc6040bb805bb3ffc8a8a871 already in progress: creating tablet
I20260505 07:53:58.866050 1171 ts_tablet_manager.cc:1916] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.869343 1178 ts_tablet_manager.cc:1929] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.874053 1178 log.cc:1199] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/956b5e05bda648349ba26d5c1ea3b279
I20260505 07:53:58.874584 1178 ts_tablet_manager.cc:1950] T 956b5e05bda648349ba26d5c1ea3b279 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:58.878120 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 245) took 1352 ms. Trace:
I20260505 07:53:58.878304 1178 rpcz_store.cc:276] 0505 07:53:57.525890 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.526021 (+ 131us) service_pool.cc:224] Handling call
0505 07:53:58.810220 (+1284199us) ts_tablet_manager.cc:1151] Deleting tablet 956b5e05bda648349ba26d5c1ea3b279
0505 07:53:58.810235 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:58.848150 (+ 37915us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.869261 (+ 21111us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.878044 (+ 8783us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1283962}
I20260505 07:53:58.879071 1203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:53:58.879262 1203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: This master's current role is: LEADER
I20260505 07:53:58.880019 1178 tablet_replica.cc:333] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:58.881865 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 956b5e05bda648349ba26d5c1ea3b279 (table default.Table_017976a3a6af42c9a9fc722661286c0e [id=d524458d873a48cb980e2f85490232d6]) successfully deleted
I20260505 07:53:58.882421 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_fd75ee1f117344169ccafd58dc18408f" } modify_external_catalogs: true
I20260505 07:53:58.882696 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_fd75ee1f117344169ccafd58dc18408f" } modify_external_catalogs: true
I20260505 07:53:58.880273 1178 raft_consensus.cc:2243] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.887116 1178 raft_consensus.cc:2272] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.888900 1172 tablet_bootstrap.cc:492] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:58.890900 1172 ts_tablet_manager.cc:1403] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.070s user 0.006s sys 0.000s
I20260505 07:53:58.891927 1172 raft_consensus.cc:359] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:58.892102 1172 raft_consensus.cc:385] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.892184 1172 raft_consensus.cc:740] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.892410 1172 consensus_queue.cc:260] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:58.892916 1172 ts_tablet_manager.cc:1434] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:53:58.893235 1172 tablet_bootstrap.cc:492] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:58.887683 1178 ts_tablet_manager.cc:1916] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.897920 1172 tablet_bootstrap.cc:654] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.908985 1357 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" instance_seqno: 1777967632042903) as {username='slave'} at 127.0.105.2:57579; Asking this server to re-register.
I20260505 07:53:58.910126 971 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:58.914755 1178 ts_tablet_manager.cc:1929] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.914906 1178 log.cc:1199] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/ce9d01b0b5464a65966f26839b48aaf1
I20260505 07:53:58.915413 1178 ts_tablet_manager.cc:1950] T ce9d01b0b5464a65966f26839b48aaf1 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:58.915947 1518 leader_election.cc:290] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 379cdfdecc3341ea9d7d9dc09274338d (127.0.105.62:33621), 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401)
I20260505 07:53:58.917709 1171 ts_tablet_manager.cc:1929] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:53:58.917862 1171 log.cc:1199] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/3781d0bbcd1b4189bd836eeae4236f40
I20260505 07:53:58.918426 1176 tablet_bootstrap.cc:654] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.923465 1358 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" instance_seqno: 1777967632532909) as {username='slave'} at 127.0.105.3:45737; Asking this server to re-register.
I20260505 07:53:58.924737 1102 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:58.926457 1171 ts_tablet_manager.cc:1950] T 3781d0bbcd1b4189bd836eeae4236f40 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:58.927191 1357 ts_manager.cc:194] Registered new tserver with Master: d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:53:58.927654 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]) successfully deleted
I20260505 07:53:58.929467 1358 ts_manager.cc:194] Registered new tserver with Master: d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:53:58.929455 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 246) took 1402 ms. Trace:
I20260505 07:53:58.929581 1178 rpcz_store.cc:276] 0505 07:53:57.527136 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.527264 (+ 128us) service_pool.cc:224] Handling call
0505 07:53:58.879901 (+1352637us) ts_tablet_manager.cc:1151] Deleting tablet ce9d01b0b5464a65966f26839b48aaf1
0505 07:53:58.879914 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:58.903073 (+ 23159us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.914644 (+ 11571us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.929413 (+ 14769us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1352435}
I20260505 07:53:58.929924 1178 tablet_replica.cc:333] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:58.930140 1178 raft_consensus.cc:2243] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.930303 1178 raft_consensus.cc:2272] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.930354 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet ce9d01b0b5464a65966f26839b48aaf1 (table default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]) successfully deleted
I20260505 07:53:58.930855 1178 ts_tablet_manager.cc:1916] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.942116 1358 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" instance_seqno: 1777967631556394) as {username='slave'} at 127.0.105.1:36755; Asking this server to re-register.
I20260505 07:53:58.942924 829 heartbeater.cc:461] Registering TS with master...
I20260505 07:53:58.945063 1357 ts_manager.cc:194] Registered new tserver with Master: eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:58.945775 1175 ts_tablet_manager.cc:1916] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.948318 1172 tablet_bootstrap.cc:492] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:58.948670 1172 ts_tablet_manager.cc:1403] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.001s
I20260505 07:53:58.949536 1172 raft_consensus.cc:359] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:58.949818 1172 raft_consensus.cc:385] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.949945 1172 raft_consensus.cc:740] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.950403 1172 consensus_queue.cc:260] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:58.950601 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 8ca4cf248adb4f768c5586d72614ad42
I20260505 07:53:58.951017 1172 ts_tablet_manager.cc:1434] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:53:58.951123 598 catalog_manager.cc:6741] ReplaceTablet: tablet 8ca4cf248adb4f768c5586d72614ad42 deleted and replaced by tablet 423c22891bed462d89e9ec51a079f0f5
I20260505 07:53:58.951953 893 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ca4cf248adb4f768c5586d72614ad42 with delete_type TABLET_DATA_DELETED (replaced by tablet 423c22891bed462d89e9ec51a079f0f5) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:58.952455 1171 tablet_replica.cc:333] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:58.952862 1171 raft_consensus.cc:2243] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:58.953125 1171 raft_consensus.cc:2272] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.953020 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_c12be1928760438fab7c135df431bc79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.953570 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c12be1928760438fab7c135df431bc79 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.954455 1171 ts_tablet_manager.cc:1916] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.955436 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_37add05ae60c4b40b655ffe723848dfd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:58.955968 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37add05ae60c4b40b655ffe723848dfd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:58.957170 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ca4cf248adb4f768c5586d72614ad42 with delete_type TABLET_DATA_DELETED (replaced by tablet 423c22891bed462d89e9ec51a079f0f5) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:58.960155 1172 tablet_bootstrap.cc:492] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:58.964792 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 8ca4cf248adb4f768c5586d72614ad42 with delete_type TABLET_DATA_DELETED (replaced by tablet 423c22891bed462d89e9ec51a079f0f5) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:58.970439 1178 ts_tablet_manager.cc:1929] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:58.971037 1176 tablet_bootstrap.cc:492] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:58.971271 1176 ts_tablet_manager.cc:1403] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.123s user 0.002s sys 0.001s
I20260505 07:53:58.971287 1178 log.cc:1199] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/5b124eda66364b7a81426934b98bca48
I20260505 07:53:58.971810 1178 ts_tablet_manager.cc:1950] T 5b124eda66364b7a81426934b98bca48 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:58.976387 1172 tablet_bootstrap.cc:654] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.978760 1176 raft_consensus.cc:359] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.979007 1176 raft_consensus.cc:385] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.979086 1176 raft_consensus.cc:740] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.979317 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 247) took 1451 ms. Trace:
I20260505 07:53:58.979333 1176 consensus_queue.cc:260] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:58.980839 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0e4e5af08ddc4c77b5640b8295b236b1 consensus state has no leader
I20260505 07:53:58.979465 1178 rpcz_store.cc:276] 0505 07:53:57.528342 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.528467 (+ 125us) service_pool.cc:224] Handling call
0505 07:53:58.929860 (+1401393us) ts_tablet_manager.cc:1151] Deleting tablet 5b124eda66364b7a81426934b98bca48
0505 07:53:58.929870 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:58.951344 (+ 21474us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.970354 (+ 19010us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:58.979261 (+ 8907us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1401205}
I20260505 07:53:58.981907 1178 tablet_replica.cc:333] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:58.982102 1178 raft_consensus.cc:2243] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:53:58.982893 1178 pending_rounds.cc:70] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:53:58.982964 1178 pending_rounds.cc:77] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555442799046656 op_type: NO_OP noop_request { }
I20260505 07:53:58.983086 1178 raft_consensus.cc:2889] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:53:58.983184 1178 raft_consensus.cc:2272] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:58.983798 1178 ts_tablet_manager.cc:1916] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:58.983939 1176 ts_tablet_manager.cc:1434] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260505 07:53:58.984319 1176 tablet_bootstrap.cc:492] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:58.985618 1181 tablet_bootstrap.cc:492] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:58.985790 1181 ts_tablet_manager.cc:1403] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.123s user 0.013s sys 0.011s
I20260505 07:53:58.986163 762 tablet_service.cc:1511] Processing CreateTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e (DEFAULT_TABLE table=default.Table_ff8ae2444af4400eb86af8cc0fa3c146 [id=6a242917efbf4ae5a5fa82539d6f7b34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.986575 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.986567 1181 raft_consensus.cc:359] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.986759 1181 raft_consensus.cc:385] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:58.986840 1181 raft_consensus.cc:740] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:58.987073 1181 consensus_queue.cc:260] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:58.988790 597 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (table default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c [id=8a910f6a553a47ea87ec170e376982d1]). Expected version 2 got 0
I20260505 07:53:58.989485 1181 ts_tablet_manager.cc:1434] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:53:58.989519 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 7001482d90364dbdb8b9c5bfb98e1cd4 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:58.990659 1181 tablet_bootstrap.cc:492] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:58.990873 893 tablet_service.cc:1511] Processing CreateTablet for tablet 3f869abaa31e414ea0625e475db08e64 (DEFAULT_TABLE table=default.Table_c1551c1a06c34c5dae05b3d5f1109d43 [id=276ccf1ef8a9474dbb936d880f10805e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.991323 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f869abaa31e414ea0625e475db08e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.991608 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 3f869abaa31e414ea0625e475db08e64 (DEFAULT_TABLE table=default.Table_c1551c1a06c34c5dae05b3d5f1109d43 [id=276ccf1ef8a9474dbb936d880f10805e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.993371 1026 tablet_service.cc:1511] Processing CreateTablet for tablet b5a0cbf9c3d74cea92920189da332c1c (DEFAULT_TABLE table=default.Table_4d333714e70b4e53b695681972d14faf [id=f99b727e2aa54591baf6f109c07c2677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.994472 1176 tablet_bootstrap.cc:654] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.995617 1022 tablet_service.cc:1511] Processing CreateTablet for tablet c837877946604ef29f95dac9dd449bf3 (DEFAULT_TABLE table=default.Table_7c3d336ba1db41b9b26a2fe5475bbab0 [id=50b5f277bb144f4893d158d179bd0fb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.996065 901 tablet_service.cc:1511] Processing CreateTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e (DEFAULT_TABLE table=default.Table_ff8ae2444af4400eb86af8cc0fa3c146 [id=6a242917efbf4ae5a5fa82539d6f7b34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.996536 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:58.997684 1017 tablet_service.cc:1511] Processing CreateTablet for tablet b256d072933b4566b5587224274deb66 (DEFAULT_TABLE table=default.Table_6cb216f92da74a78acd72a6132b94c1a [id=c0306ceddf024a1589f83f390b38c602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:58.997910 1181 tablet_bootstrap.cc:654] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:58.999892 895 tablet_service.cc:1511] Processing CreateTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf (DEFAULT_TABLE table=default.Table_314825f5a7f644dba828c34f3d2d740a [id=056371ab05fc41c78f31a8ac6887ecb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.000240 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7e0c8ad716e443588d4ae3aa28ed4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.002666 887 tablet_service.cc:1511] Processing CreateTablet for tablet f106a6d4855148fe83b1e90ea9f27d39 (DEFAULT_TABLE table=default.Table_4a5f2bded41d47d59bd0da9575fb955a [id=646a36b5d2b64ae3836951b29ae9ac00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.003207 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e (DEFAULT_TABLE table=default.Table_ff8ae2444af4400eb86af8cc0fa3c146 [id=6a242917efbf4ae5a5fa82539d6f7b34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.003671 1025 tablet_service.cc:1511] Processing CreateTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf (DEFAULT_TABLE table=default.Table_314825f5a7f644dba828c34f3d2d740a [id=056371ab05fc41c78f31a8ac6887ecb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.004104 1032 tablet_service.cc:1511] Processing CreateTablet for tablet f106a6d4855148fe83b1e90ea9f27d39 (DEFAULT_TABLE table=default.Table_4a5f2bded41d47d59bd0da9575fb955a [id=646a36b5d2b64ae3836951b29ae9ac00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.005178 1035 tablet_service.cc:1511] Processing CreateTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e (DEFAULT_TABLE table=default.Table_7f7ad7a477f249319d0428cd775dc541 [id=4a85ae577239475da68f2677a043845e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.005590 1027 tablet_service.cc:1511] Processing CreateTablet for tablet abe5f6f0774545b296bf1b03a08e145d (DEFAULT_TABLE table=default.Table_c22dca26148c48f882eeea850f50bea4 [id=6c12928213864a1195f14e35d819f339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.006026 1028 tablet_service.cc:1511] Processing CreateTablet for tablet aa049ad4edbd466a860428c822a52724 (DEFAULT_TABLE table=default.Table_881e42cce93c46cb8c6b2ff28d713b85 [id=bc1adc573d4c43afa8c5bdd900e53d65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.006639 467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23c57a2c927c4c54bccecfeb3a775311" candidate_term: 2 candidate_status { last_received { term: 1 index: 373 } } ignore_live_leader: false dest_uuid: "379cdfdecc3341ea9d7d9dc09274338d" is_pre_election: true
I20260505 07:53:59.010121 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f869abaa31e414ea0625e475db08e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.015149 757 tablet_service.cc:1511] Processing CreateTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf (DEFAULT_TABLE table=default.Table_314825f5a7f644dba828c34f3d2d740a [id=056371ab05fc41c78f31a8ac6887ecb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.015837 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a0cbf9c3d74cea92920189da332c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.015877 760 tablet_service.cc:1511] Processing CreateTablet for tablet f106a6d4855148fe83b1e90ea9f27d39 (DEFAULT_TABLE table=default.Table_4a5f2bded41d47d59bd0da9575fb955a [id=646a36b5d2b64ae3836951b29ae9ac00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.016340 749 tablet_service.cc:1511] Processing CreateTablet for tablet b5a0cbf9c3d74cea92920189da332c1c (DEFAULT_TABLE table=default.Table_4d333714e70b4e53b695681972d14faf [id=f99b727e2aa54591baf6f109c07c2677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.016880 751 tablet_service.cc:1511] Processing CreateTablet for tablet c837877946604ef29f95dac9dd449bf3 (DEFAULT_TABLE table=default.Table_7c3d336ba1db41b9b26a2fe5475bbab0 [id=50b5f277bb144f4893d158d179bd0fb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.016975 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_09561894665e45a98f269fd1d764b78a" } new_table_name: "default.Table_589d610a49c248009e067d71c961eef9" modify_external_catalogs: true
I20260505 07:53:59.017395 754 tablet_service.cc:1511] Processing CreateTablet for tablet b256d072933b4566b5587224274deb66 (DEFAULT_TABLE table=default.Table_6cb216f92da74a78acd72a6132b94c1a [id=c0306ceddf024a1589f83f390b38c602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.017894 744 tablet_service.cc:1511] Processing CreateTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e (DEFAULT_TABLE table=default.Table_7f7ad7a477f249319d0428cd775dc541 [id=4a85ae577239475da68f2677a043845e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.018370 746 tablet_service.cc:1511] Processing CreateTablet for tablet abe5f6f0774545b296bf1b03a08e145d (DEFAULT_TABLE table=default.Table_c22dca26148c48f882eeea850f50bea4 [id=6c12928213864a1195f14e35d819f339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.018872 755 tablet_service.cc:1511] Processing CreateTablet for tablet aa049ad4edbd466a860428c822a52724 (DEFAULT_TABLE table=default.Table_881e42cce93c46cb8c6b2ff28d713b85 [id=bc1adc573d4c43afa8c5bdd900e53d65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.019363 761 tablet_service.cc:1511] Processing CreateTablet for tablet a7d8265c05b74b28959d0de41fad1fde (DEFAULT_TABLE table=default.Table_4e8ee4d80a174660827b921c9f211ac4 [id=ce8bd78012db468e8a358f51c308a86e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.020272 1031 tablet_service.cc:1511] Processing CreateTablet for tablet a7d8265c05b74b28959d0de41fad1fde (DEFAULT_TABLE table=default.Table_4e8ee4d80a174660827b921c9f211ac4 [id=ce8bd78012db468e8a358f51c308a86e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.020669 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7d8265c05b74b28959d0de41fad1fde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.023247 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_433dccceaf294d6facc01e2ea5297a9f" } new_table_name: "default.Table_ceaf4d54d57542928716e3056eac2bde" modify_external_catalogs: true
I20260505 07:53:59.024513 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c837877946604ef29f95dac9dd449bf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.026006 611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23c57a2c927c4c54bccecfeb3a775311" candidate_term: 2 candidate_status { last_received { term: 1 index: 373 } } ignore_live_leader: false dest_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" is_pre_election: true
I20260505 07:53:59.027709 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet c7652191b1d44ba59d856a2b0d7017b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.027832 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b256d072933b4566b5587224274deb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.028699 1178 ts_tablet_manager.cc:1929] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:53:59.028858 1178 log.cc:1199] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/8cfe212cb6c14ce3a63b6ee4275f66f1
I20260505 07:53:59.029263 1178 ts_tablet_manager.cc:1950] T 8cfe212cb6c14ce3a63b6ee4275f66f1 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:59.030159 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 248) took 1501 ms. Trace:
I20260505 07:53:59.030256 1178 rpcz_store.cc:276] 0505 07:53:57.529377 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.529498 (+ 121us) service_pool.cc:224] Handling call
0505 07:53:58.981694 (+1452196us) ts_tablet_manager.cc:1151] Deleting tablet 8cfe212cb6c14ce3a63b6ee4275f66f1
0505 07:53:58.981817 (+ 123us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:59.004097 (+ 22280us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:59.028627 (+ 24530us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:59.030099 (+ 1472us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1452008}
I20260505 07:53:59.031490 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 8cfe212cb6c14ce3a63b6ee4275f66f1 (table default.Table_2b834837c1d54b429b8d9a937f56d726 [id=49a5c863608b402a9ca94c01e5a5c09f]) successfully deleted
I20260505 07:53:59.032282 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.036772 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7e0c8ad716e443588d4ae3aa28ed4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.038609 1178 tablet_replica.cc:333] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:59.039104 1178 raft_consensus.cc:2243] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.039535 1178 raft_consensus.cc:2272] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.039659 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f106a6d4855148fe83b1e90ea9f27d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.039834 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_c6c7a2a777c24a65a8217641ac14941c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.040004 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0654ae2daf60435697eeca15ba0c19d7" } modify_external_catalogs: true
I20260505 07:53:59.040194 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0654ae2daf60435697eeca15ba0c19d7" } modify_external_catalogs: true
I20260505 07:53:59.040377 1178 ts_tablet_manager.cc:1916] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:59.040393 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c6c7a2a777c24a65a8217641ac14941c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.041261 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_a86abb495238445381ac0202804b4ca0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.041697 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a86abb495238445381ac0202804b4ca0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.045766 896 tablet_service.cc:1511] Processing CreateTablet for tablet b5a0cbf9c3d74cea92920189da332c1c (DEFAULT_TABLE table=default.Table_4d333714e70b4e53b695681972d14faf [id=f99b727e2aa54591baf6f109c07c2677]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.046492 882 tablet_service.cc:1511] Processing CreateTablet for tablet abe5f6f0774545b296bf1b03a08e145d (DEFAULT_TABLE table=default.Table_c22dca26148c48f882eeea850f50bea4 [id=6c12928213864a1195f14e35d819f339]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.046520 884 tablet_service.cc:1511] Processing CreateTablet for tablet c837877946604ef29f95dac9dd449bf3 (DEFAULT_TABLE table=default.Table_7c3d336ba1db41b9b26a2fe5475bbab0 [id=50b5f277bb144f4893d158d179bd0fb0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.046885 886 tablet_service.cc:1511] Processing CreateTablet for tablet aa049ad4edbd466a860428c822a52724 (DEFAULT_TABLE table=default.Table_881e42cce93c46cb8c6b2ff28d713b85 [id=bc1adc573d4c43afa8c5bdd900e53d65]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.047268 897 tablet_service.cc:1511] Processing CreateTablet for tablet a7d8265c05b74b28959d0de41fad1fde (DEFAULT_TABLE table=default.Table_4e8ee4d80a174660827b921c9f211ac4 [id=ce8bd78012db468e8a358f51c308a86e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.048296 883 tablet_service.cc:1511] Processing CreateTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e (DEFAULT_TABLE table=default.Table_7f7ad7a477f249319d0428cd775dc541 [id=4a85ae577239475da68f2677a043845e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.050952 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7e0c8ad716e443588d4ae3aa28ed4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.054827 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abe5f6f0774545b296bf1b03a08e145d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.055725 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c837877946604ef29f95dac9dd449bf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.056043 1175 ts_tablet_manager.cc:1929] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:53:59.056948 1175 log.cc:1199] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/3781d0bbcd1b4189bd836eeae4236f40
I20260505 07:53:59.057431 1175 ts_tablet_manager.cc:1950] T 3781d0bbcd1b4189bd836eeae4236f40 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:59.057715 1425 raft_consensus.cc:493] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:59.057879 1425 raft_consensus.cc:515] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.058954 1524 catalog_manager.cc:1269] Loaded cluster ID: 9b9d7d3d45334a3e97a4e71f5c281d3c
I20260505 07:53:59.059047 1524 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: loading cluster ID for follower catalog manager: success
I20260505 07:53:59.060340 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7d8265c05b74b28959d0de41fad1fde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.062951 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7fe9c7199cc4039ae877dc3d4aab17e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.066164 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa049ad4edbd466a860428c822a52724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.068324 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b256d072933b4566b5587224274deb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.069768 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0bda6aa949274f898514caa0a79767d3" } new_table_name: "default.Table_0393e87d0a314ee898b11935e2f6666d" modify_external_catalogs: true
I20260505 07:53:59.071425 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a0cbf9c3d74cea92920189da332c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.071705 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f106a6d4855148fe83b1e90ea9f27d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.038760 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f106a6d4855148fe83b1e90ea9f27d39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.079241 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 451e99a0dfec4823812e5991147b132e with delete_type TABLET_DATA_DELETED (replaced by tablet fcd36b5a84b4442fb76e1610d8af4c07) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.079558 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "451e99a0dfec4823812e5991147b132e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:59.079685 758 tablet_service.cc:1511] Processing CreateTablet for tablet 3f869abaa31e414ea0625e475db08e64 (DEFAULT_TABLE table=default.Table_c1551c1a06c34c5dae05b3d5f1109d43 [id=276ccf1ef8a9474dbb936d880f10805e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.079813 781 raft_consensus.cc:2468] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:53:59.080061 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f869abaa31e414ea0625e475db08e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.085604 1317 leader_election.cc:304] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23c57a2c927c4c54bccecfeb3a775311; no voters: 28d3eda04f9c47ae976d19c218b6dcc5, 379cdfdecc3341ea9d7d9dc09274338d
I20260505 07:53:59.087364 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 727cacd242844455915bd9963d2db19f
I20260505 07:53:59.087792 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 2d86b8b4dadf4fd2b6409a8cb70e313e
I20260505 07:53:59.088263 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet e29ba55a846548679cb3b86305d0d6ed
I20260505 07:53:59.088979 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 1852f7f4ff594639908b03730b68fe86 from {username='slave'} at 127.0.0.1:47238
W20260505 07:53:59.089562 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1001dd888ecd4166ba217506739f71e8 consensus state has no leader
I20260505 07:53:59.090574 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.090803 756 tablet_service.cc:1558] Processing DeleteTablet for tablet e29ba55a846548679cb3b86305d0d6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.091115 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 727cacd242844455915bd9963d2db19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.094828 1518 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:53:59.097570 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c837877946604ef29f95dac9dd449bf3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.098664 1175 tablet_replica.cc:333] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:59.098881 1175 raft_consensus.cc:2243] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.099038 1175 raft_consensus.cc:2272] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.099838 1175 ts_tablet_manager.cc:1916] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.100023 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7d8265c05b74b28959d0de41fad1fde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.100996 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 727cacd242844455915bd9963d2db19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.101842 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7fe9c7199cc4039ae877dc3d4aab17e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.102555 988 leader_election.cc:304] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:53:59.103786 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa049ad4edbd466a860428c822a52724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.104554 1425 leader_election.cc:290] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:59.107057 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7fe9c7199cc4039ae877dc3d4aab17e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.110286 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]) successfully deleted
I20260505 07:53:59.110956 898 tablet_service.cc:1511] Processing CreateTablet for tablet b256d072933b4566b5587224274deb66 (DEFAULT_TABLE table=default.Table_6cb216f92da74a78acd72a6132b94c1a [id=c0306ceddf024a1589f83f390b38c602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.112903 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.113054 1425 raft_consensus.cc:2804] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:59.114173 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b64f571bfd004207bd2fb763c53b9502" } new_table_name: "default.Table_6bd554849539457bb38dbbef32deb609" modify_external_catalogs: true
I20260505 07:53:59.116382 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_2f8e689cd77249e9a31aa6d90f55b4ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.117030 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2f8e689cd77249e9a31aa6d90f55b4ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.117511 1175 ts_tablet_manager.cc:1929] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.117630 1175 log.cc:1199] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/8573f8bd424641438e2aa6ff6869180e
I20260505 07:53:59.118022 1175 ts_tablet_manager.cc:1950] T 8573f8bd424641438e2aa6ff6869180e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:59.118686 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_314825f5a7f644dba828c34f3d2d740a" } modify_external_catalogs: true
I20260505 07:53:59.118865 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_314825f5a7f644dba828c34f3d2d740a" } modify_external_catalogs: true
I20260505 07:53:59.120324 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7e8b9279f3e244a99b8ba4a25e7c569f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.120855 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7e8b9279f3e244a99b8ba4a25e7c569f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.124604 1175 tablet_replica.cc:333] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:59.124807 1175 raft_consensus.cc:2243] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:53:59.125146 1175 raft_consensus.cc:2272] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.125685 1175 ts_tablet_manager.cc:1916] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.134918 1425 raft_consensus.cc:493] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:59.135079 1425 raft_consensus.cc:3060] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:59.135869 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet e29ba55a846548679cb3b86305d0d6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.137059 1176 tablet_bootstrap.cc:492] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:59.137202 1176 ts_tablet_manager.cc:1403] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.153s user 0.006s sys 0.000s
I20260505 07:53:59.137925 1176 raft_consensus.cc:359] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.138085 1176 raft_consensus.cc:385] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.138155 1176 raft_consensus.cc:740] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.138351 1176 consensus_queue.cc:260] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.138768 1176 ts_tablet_manager.cc:1434] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:53:59.139977 1176 tablet_bootstrap.cc:492] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:59.140666 1175 ts_tablet_manager.cc:1929] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260505 07:53:59.140805 1175 log.cc:1199] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/8ca4cf248adb4f768c5586d72614ad42
I20260505 07:53:59.141263 1175 ts_tablet_manager.cc:1950] T 8ca4cf248adb4f768c5586d72614ad42 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:59.141881 1176 tablet_bootstrap.cc:654] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.142097 1175 tablet_replica.cc:333] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:59.142273 1175 raft_consensus.cc:2243] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.142463 1175 raft_consensus.cc:2272] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.149953 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 8573f8bd424641438e2aa6ff6869180e (table default.Table_7d1be5262b41414f8a0fc37fba1cc28a [id=128f12d4ca8c43f6887ae3240610ef60]) successfully deleted
I20260505 07:53:59.150583 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]) successfully deleted
I20260505 07:53:59.160171 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 7001482d90364dbdb8b9c5bfb98e1cd4
I20260505 07:53:59.160713 597 catalog_manager.cc:6741] ReplaceTablet: tablet 7001482d90364dbdb8b9c5bfb98e1cd4 deleted and replaced by tablet 6f207a3ffbc94b01abd9e9d6837f5c81
I20260505 07:53:59.164156 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_89965306827247e99ff9f34d13e63c54"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.164429 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f207a3ffbc94b01abd9e9d6837f5c81) from {username='slave'} at 127.0.0.1:45248
W20260505 07:53:59.164853 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89965306827247e99ff9f34d13e63c54 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.165494 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abe5f6f0774545b296bf1b03a08e145d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.167296 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b256d072933b4566b5587224274deb66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.167639 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abe5f6f0774545b296bf1b03a08e145d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.167713 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 727cacd242844455915bd9963d2db19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.168372 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.168958 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5a0cbf9c3d74cea92920189da332c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.170492 890 tablet_service.cc:1558] Processing DeleteTablet for tablet e29ba55a846548679cb3b86305d0d6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:53:59.174636 991 leader_election.cc:343] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 451e99a0dfec4823812e5991147b132e
I20260505 07:53:59.175897 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f207a3ffbc94b01abd9e9d6837f5c81) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.176110 1524 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: acquiring CA information for follower catalog manager: success
I20260505 07:53:59.176554 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 872362aecf6a44b59e586b1fa7637c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.177330 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 202a237ca580482c98803683a77d96d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.177675 744 tablet_service.cc:1558] Processing DeleteTablet for tablet c7652191b1d44ba59d856a2b0d7017b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.177640 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f207a3ffbc94b01abd9e9d6837f5c81) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.178040 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa049ad4edbd466a860428c822a52724. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:59.182705 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 100526us
W20260505 07:53:59.188504 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 54e4f7f895d943639653c8c32724baf2 consensus state has no leader
I20260505 07:53:59.189392 1175 ts_tablet_manager.cc:1916] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.191141 601 master_service.cc:954] ReplaceTablet: received request to replace tablet 46274a44a06b4306b5dac43c66c85a80 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:59.192268 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f6734bb37ebf474982961491f124f0f1" } new_table_name: "default.Table_575ab5b128ee4a1baebbe1f90baf005a" modify_external_catalogs: true
I20260505 07:53:59.201699 1175 ts_tablet_manager.cc:1929] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.201850 1175 log.cc:1199] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/451e99a0dfec4823812e5991147b132e
I20260505 07:53:59.202260 1175 ts_tablet_manager.cc:1950] T 451e99a0dfec4823812e5991147b132e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:59.203359 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 451e99a0dfec4823812e5991147b132e (table default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95]) successfully deleted
W20260505 07:53:59.204052 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d86b8b4dadf4fd2b6409a8cb70e313e already in progress: creating tablet
W20260505 07:53:59.204634 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e29ba55a846548679cb3b86305d0d6ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e29ba55a846548679cb3b86305d0d6ed already in progress: creating tablet
W20260505 07:53:59.205164 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 727cacd242844455915bd9963d2db19f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 727cacd242844455915bd9963d2db19f already in progress: creating tablet
I20260505 07:53:59.205452 1175 tablet_replica.cc:333] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:59.205634 1175 raft_consensus.cc:2243] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.205785 1175 raft_consensus.cc:2272] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.206775 1175 ts_tablet_manager.cc:1916] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.213361 1176 tablet_bootstrap.cc:492] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:59.213516 1176 ts_tablet_manager.cc:1403] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.074s user 0.003s sys 0.000s
I20260505 07:53:59.214290 1176 raft_consensus.cc:359] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.214483 1176 raft_consensus.cc:385] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.214555 1176 raft_consensus.cc:740] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.214809 1176 consensus_queue.cc:260] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.215268 1176 ts_tablet_manager.cc:1434] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:53:59.216899 1176 tablet_bootstrap.cc:492] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:59.218981 1176 tablet_bootstrap.cc:654] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.221971 1425 raft_consensus.cc:515] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.222579 1425 leader_election.cc:290] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:53:59.225040 991 leader_election.cc:343] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 451e99a0dfec4823812e5991147b132e
W20260505 07:53:59.225267 991 consensus_peers.cc:597] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3781d0bbcd1b4189bd836eeae4236f40. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:53:59.226207 988 consensus_peers.cc:597] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3781d0bbcd1b4189bd836eeae4236f40. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:53:59.226819 988 leader_election.cc:343] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 451e99a0dfec4823812e5991147b132e
I20260505 07:53:59.226950 988 leader_election.cc:304] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:59.229210 1425 raft_consensus.cc:2749] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260505 07:53:59.229506 1524 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260505 07:53:59.230626 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1f4b1af520d64a108ff4289226a1561d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.230755 754 tablet_service.cc:1558] Processing DeleteTablet for tablet e1a9420a81844d2092381123d9297661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:59.231160 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f4b1af520d64a108ff4289226a1561d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.231758 1175 ts_tablet_manager.cc:1929] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.231896 1175 log.cc:1199] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/c7652191b1d44ba59d856a2b0d7017b1
I20260505 07:53:59.232300 1175 ts_tablet_manager.cc:1950] T c7652191b1d44ba59d856a2b0d7017b1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:59.233232 1171 ts_tablet_manager.cc:1929] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260505 07:53:59.233353 1171 log.cc:1199] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/8ca4cf248adb4f768c5586d72614ad42
I20260505 07:53:59.234275 1171 ts_tablet_manager.cc:1950] T 8ca4cf248adb4f768c5586d72614ad42 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:59.235509 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]) successfully deleted
I20260505 07:53:59.236251 1171 tablet_replica.cc:333] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:59.236734 1172 tablet_bootstrap.cc:492] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
W20260505 07:53:59.237241 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a9b7c6e9463342a5a73af12da4447f89 has no consensus state
W20260505 07:53:59.237612 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df1b62a30b2842b897a04c793bb5e7ce consensus state has no leader
I20260505 07:53:59.237986 1171 raft_consensus.cc:2243] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.238170 1171 raft_consensus.cc:2272] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:53:59.241472 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 727cacd242844455915bd9963d2db19f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 727cacd242844455915bd9963d2db19f already in progress: creating tablet
W20260505 07:53:59.241711 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d86b8b4dadf4fd2b6409a8cb70e313e already in progress: creating tablet
W20260505 07:53:59.241909 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e29ba55a846548679cb3b86305d0d6ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e29ba55a846548679cb3b86305d0d6ed already in progress: creating tablet
W20260505 07:53:59.242115 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7001482d90364dbdb8b9c5bfb98e1cd4 already in progress: creating tablet
I20260505 07:53:59.242832 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet d7e0c8ad716e443588d4ae3aa28ed4bf
I20260505 07:53:59.244484 754 tablet_service.cc:1558] Processing DeleteTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.245462 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet c7652191b1d44ba59d856a2b0d7017b1 (table default.Table_6e9d636b8b314e5b888c725d34775180 [id=73c0bcbf3485405589f58b8175bfc3b9]) successfully deleted
W20260505 07:53:59.245725 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7001482d90364dbdb8b9c5bfb98e1cd4 already in progress: creating tablet
I20260505 07:53:59.247336 1175 tablet_replica.cc:333] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:59.247694 1172 ts_tablet_manager.cc:1403] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.285s user 0.006s sys 0.015s
I20260505 07:53:59.247571 1175 raft_consensus.cc:2243] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.251848 1175 raft_consensus.cc:2272] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.257510 1176 tablet_bootstrap.cc:492] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:59.258942 1176 ts_tablet_manager.cc:1403] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.000s
I20260505 07:53:59.259471 1178 ts_tablet_manager.cc:1929] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.259644 1178 log.cc:1199] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/1765a95cc3414c73b7e3918913a35937
I20260505 07:53:59.259963 1178 ts_tablet_manager.cc:1950] T 1765a95cc3414c73b7e3918913a35937 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:59.260079 1176 raft_consensus.cc:359] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.260301 1176 raft_consensus.cc:385] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.260413 1176 raft_consensus.cc:740] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.260682 1176 consensus_queue.cc:260] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.261180 1176 ts_tablet_manager.cc:1434] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:53:59.257757 1175 ts_tablet_manager.cc:1916] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.262159 1176 tablet_bootstrap.cc:492] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:59.262179 893 tablet_service.cc:1558] Processing DeleteTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.262432 1171 ts_tablet_manager.cc:1916] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.263054 1172 raft_consensus.cc:359] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.263799 1172 raft_consensus.cc:385] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.265410 1172 raft_consensus.cc:740] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.265661 1172 consensus_queue.cc:260] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.266453 1172 ts_tablet_manager.cc:1434] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260505 07:53:59.266777 1172 tablet_bootstrap.cc:492] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:59.263877 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 25a74b65d5d6459293ab4461f4d34cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:59.269542 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 080b2984fcf2478a8def6540da63554d has no consensus state
I20260505 07:53:59.270460 1366 raft_consensus.cc:493] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:59.270586 1366 raft_consensus.cc:515] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.270872 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 249) took 1740 ms. Trace:
I20260505 07:53:59.271184 1366 leader_election.cc:290] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:59.271853 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d83e9d4243e409794ffe8a342bce6b3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:59.272275 1181 tablet_bootstrap.cc:492] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:59.272410 1181 ts_tablet_manager.cc:1403] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.282s user 0.031s sys 0.057s
W20260505 07:53:59.272410 854 leader_election.cc:343] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:59.270987 1178 rpcz_store.cc:276] 0505 07:53:57.530381 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.530570 (+ 189us) service_pool.cc:224] Handling call
0505 07:53:59.038505 (+1507935us) ts_tablet_manager.cc:1151] Deleting tablet 1765a95cc3414c73b7e3918913a35937
0505 07:53:59.038514 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:59.256283 (+217769us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:59.259385 (+ 3102us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:59.270834 (+ 11449us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1507729}
I20260505 07:53:59.273627 1178 tablet_replica.cc:333] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:59.274024 1178 raft_consensus.cc:2243] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.275489 1178 raft_consensus.cc:2272] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.277163 1178 ts_tablet_manager.cc:1916] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.280301 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_9201abe14fd54f8880602582b1f74cc8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.280890 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9201abe14fd54f8880602582b1f74cc8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.281724 1366 raft_consensus.cc:493] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:59.281877 1366 raft_consensus.cc:515] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.282059 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d83e9d4243e409794ffe8a342bce6b3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:59.282117 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_db40b13ba59549c6bae350382395bceb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.282457 1366 leader_election.cc:290] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:53:59.282537 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db40b13ba59549c6bae350382395bceb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.283007 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f88a5c0d0e2c436c8d8b6afc1f3c2edf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:59.284040 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f88a5c0d0e2c436c8d8b6afc1f3c2edf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:59.273224 1181 raft_consensus.cc:359] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.286083 1181 raft_consensus.cc:385] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260505 07:53:59.286214 854 leader_election.cc:343] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:59.286183 1181 raft_consensus.cc:740] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.286505 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_fe563815e7df4183a9bfe81907c3c33c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.286612 1181 consensus_queue.cc:260] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:59.286947 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe563815e7df4183a9bfe81907c3c33c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.287972 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_a44f4d613a6e47e38e9741252ef6956a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.288407 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a44f4d613a6e47e38e9741252ef6956a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.289467 1175 ts_tablet_manager.cc:1929] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.289589 1175 log.cc:1199] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/e1a9420a81844d2092381123d9297661
I20260505 07:53:59.290493 1175 ts_tablet_manager.cc:1950] T e1a9420a81844d2092381123d9297661 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:59.291110 593 master_service.cc:954] ReplaceTablet: received request to replace tablet 067c32ecccb44bca8bf9cc3f6c229326 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:59.291661 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet e1a9420a81844d2092381123d9297661 (table default.Table_422080d5c75f422fb88c26b76ffc61c4 [id=0d8de70d77c54c2491b257d09c8474c8]) successfully deleted
I20260505 07:53:59.291795 1181 ts_tablet_manager.cc:1434] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.019s user 0.001s sys 0.001s
I20260505 07:53:59.292593 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c1cf7717be07400aacdab563b4255450" } modify_external_catalogs: true
I20260505 07:53:59.292752 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c1cf7717be07400aacdab563b4255450" } modify_external_catalogs: true
I20260505 07:53:59.293236 600 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4e8ee4d80a174660827b921c9f211ac4" } modify_external_catalogs: true
I20260505 07:53:59.293366 600 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4e8ee4d80a174660827b921c9f211ac4" } modify_external_catalogs: true
I20260505 07:53:59.293819 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d4d6e315fb8340e9b629d876d050ef05" } modify_external_catalogs: true
I20260505 07:53:59.293949 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d4d6e315fb8340e9b629d876d050ef05" } modify_external_catalogs: true
I20260505 07:53:59.294452 599 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_613832387c45465197e11f83984f0304" } new_table_name: "default.Table_0555c54d8e394cb1a8509950422c9ac8" modify_external_catalogs: true
W20260505 07:53:59.297832 856 leader_election.cc:343] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:59.297962 856 leader_election.cc:304] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:59.298666 1181 tablet_bootstrap.cc:492] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:59.299368 1176 tablet_bootstrap.cc:654] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.300248 1181 tablet_bootstrap.cc:654] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.301553 1366 raft_consensus.cc:2749] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:59.301630 1178 ts_tablet_manager.cc:1929] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260505 07:53:59.304337 856 leader_election.cc:343] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:59.304539 856 leader_election.cc:304] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:59.304909 1366 raft_consensus.cc:2749] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:59.306423 1178 log.cc:1199] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/4d6490f568024510ab843ec5205a4d1a
I20260505 07:53:59.307273 1178 ts_tablet_manager.cc:1950] T 4d6490f568024510ab843ec5205a4d1a P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:59.309525 1172 tablet_bootstrap.cc:654] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.311003 594 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3d83e9d4243e409794ffe8a342bce6b3 (table default.Table_563025047e4142d4807295cff836ea99 [id=5a7033e8ecc940e49794281e421153ea]). Expected version 1 got 0
I20260505 07:53:59.316016 594 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3d83e9d4243e409794ffe8a342bce6b3 (table default.Table_563025047e4142d4807295cff836ea99 [id=5a7033e8ecc940e49794281e421153ea]). Expected version 1 got 0
I20260505 07:53:59.316668 896 tablet_service.cc:1511] Processing CreateTablet for tablet c4dd3ece5cb34ceb82feeb67e75da0b5 (DEFAULT_TABLE table=default.Table_d4c7087457154e89b20abe713355e422 [id=7099b9d0095f4bf880cff569eeeb5d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.320009 1541 raft_consensus.cc:493] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:59.320138 1541 raft_consensus.cc:515] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.320766 1541 leader_election.cc:290] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:59.320919 754 tablet_service.cc:1511] Processing CreateTablet for tablet c4dd3ece5cb34ceb82feeb67e75da0b5 (DEFAULT_TABLE table=default.Table_d4c7087457154e89b20abe713355e422 [id=7099b9d0095f4bf880cff569eeeb5d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.321345 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4dd3ece5cb34ceb82feeb67e75da0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.321601 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4dd3ece5cb34ceb82feeb67e75da0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.321820 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcbebddcdc4ad9a788c01134e1101c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:59.322345 991 leader_election.cc:343] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: c6dcbebddcdc4ad9a788c01134e1101c
W20260505 07:53:59.322728 988 leader_election.cc:343] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:59.322841 988 leader_election.cc:304] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:59.323565 1541 raft_consensus.cc:2749] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:59.328027 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 250) took 1796 ms. Trace:
W20260505 07:53:59.328573 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet d7e0c8ad716e443588d4ae3aa28ed4bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d7e0c8ad716e443588d4ae3aa28ed4bf already in progress: creating tablet
I20260505 07:53:59.328737 1178 rpcz_store.cc:276] 0505 07:53:57.531661 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.531840 (+ 179us) service_pool.cc:224] Handling call
0505 07:53:59.273567 (+1741727us) ts_tablet_manager.cc:1151] Deleting tablet 4d6490f568024510ab843ec5205a4d1a
0505 07:53:59.273576 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:59.293411 (+ 19835us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:59.301570 (+ 8159us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:59.327976 (+ 26406us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1741532}
I20260505 07:53:59.328868 1175 tablet_replica.cc:333] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:59.329030 1175 raft_consensus.cc:2243] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.329190 1175 raft_consensus.cc:2272] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.329751 1175 ts_tablet_manager.cc:1916] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.329937 1178 tablet_replica.cc:333] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:59.330164 1178 raft_consensus.cc:2243] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.330387 1178 raft_consensus.cc:2272] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.330394 754 tablet_service.cc:1511] Processing CreateTablet for tablet 02e06d8815034d91986cef12964860ea (DEFAULT_TABLE table=default.Table_ca13a11f3fd249eabadc7213e7318acd [id=76602762055f42c0ad9805377eeb5d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.330780 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e06d8815034d91986cef12964860ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.331354 1178 ts_tablet_manager.cc:1916] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.338044 597 master_service.cc:954] ReplaceTablet: received request to replace tablet ba616702dc6040bb805bb3ffc8a8a871 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:59.340677 1175 ts_tablet_manager.cc:1929] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.340831 1175 log.cc:1199] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/25a74b65d5d6459293ab4461f4d34cf2
I20260505 07:53:59.341198 1175 ts_tablet_manager.cc:1950] T 25a74b65d5d6459293ab4461f4d34cf2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:59.342092 886 tablet_service.cc:1511] Processing CreateTablet for tablet 02e06d8815034d91986cef12964860ea (DEFAULT_TABLE table=default.Table_ca13a11f3fd249eabadc7213e7318acd [id=76602762055f42c0ad9805377eeb5d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.342464 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e06d8815034d91986cef12964860ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.342599 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 25a74b65d5d6459293ab4461f4d34cf2 (table default.Table_eb39510acd7a40b6a7f20bd09e300ff7 [id=94a6d460807e43768c78bc319239b699]) successfully deleted
I20260505 07:53:59.343868 751 tablet_service.cc:1511] Processing CreateTablet for tablet b6607b7428044e428af9d765c9c9b3ba (DEFAULT_TABLE table=default.Table_6f171a1e79d24e57acfe3aef43b24c7e [id=35a56d340665450ab03ba49b845ae78a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.344223 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6607b7428044e428af9d765c9c9b3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.344513 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 1765a95cc3414c73b7e3918913a35937 (table default.Table_a006de2dda30423f83e51da1e0477f9d [id=13ca7143dd0249d597c79bbfc17ae7f9]) successfully deleted
I20260505 07:53:59.345407 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 4d6490f568024510ab843ec5205a4d1a (table default.Table_ae9bf3e293014ec781c8885d8a716e83 [id=9c204b77f3df4bf9a58508967c89c9f4]) successfully deleted
I20260505 07:53:59.347019 1028 tablet_service.cc:1558] Processing DeleteTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.348420 1027 tablet_service.cc:1511] Processing CreateTablet for tablet c4dd3ece5cb34ceb82feeb67e75da0b5 (DEFAULT_TABLE table=default.Table_d4c7087457154e89b20abe713355e422 [id=7099b9d0095f4bf880cff569eeeb5d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.348567 744 tablet_service.cc:1511] Processing CreateTablet for tablet 3462dcc6391741e59dcb5cd08527e50f (DEFAULT_TABLE table=default.Table_13e8ecc0f18d4b909d161b6b56661449 [id=509c7b9d25aa48af9dde3825cfe313d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.348892 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3462dcc6391741e59dcb5cd08527e50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.349022 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4dd3ece5cb34ceb82feeb67e75da0b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.349246 1028 tablet_service.cc:1558] Processing DeleteTablet for tablet c6dcbebddcdc4ad9a788c01134e1101c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.350092 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 02e06d8815034d91986cef12964860ea (DEFAULT_TABLE table=default.Table_ca13a11f3fd249eabadc7213e7318acd [id=76602762055f42c0ad9805377eeb5d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.350435 1032 tablet_service.cc:1511] Processing CreateTablet for tablet b6607b7428044e428af9d765c9c9b3ba (DEFAULT_TABLE table=default.Table_6f171a1e79d24e57acfe3aef43b24c7e [id=35a56d340665450ab03ba49b845ae78a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.350761 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6607b7428044e428af9d765c9c9b3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.342988 1172 tablet_bootstrap.cc:492] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:59.351145 1172 ts_tablet_manager.cc:1403] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.084s user 0.004s sys 0.000s
I20260505 07:53:59.351910 1172 raft_consensus.cc:359] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.352123 1172 raft_consensus.cc:385] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.352190 1172 raft_consensus.cc:740] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.352547 1172 consensus_queue.cc:260] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.353361 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 3462dcc6391741e59dcb5cd08527e50f (DEFAULT_TABLE table=default.Table_13e8ecc0f18d4b909d161b6b56661449 [id=509c7b9d25aa48af9dde3825cfe313d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.354710 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 02412333633a4cdebe95a144fa6c363c
W20260505 07:53:59.354809 599 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4783dd0b2fbe40219852d850fc0d9a82 consensus state has no leader
I20260505 07:53:59.355242 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 067c32ecccb44bca8bf9cc3f6c229326
I20260505 07:53:59.355680 593 catalog_manager.cc:6741] ReplaceTablet: tablet 067c32ecccb44bca8bf9cc3f6c229326 deleted and replaced by tablet d5083f98e8e14ccf99a19cdf2bc59018
I20260505 07:53:59.355865 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 02412333633a4cdebe95a144fa6c363c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.356781 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet 02412333633a4cdebe95a144fa6c363c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.357528 1172 ts_tablet_manager.cc:1434] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:53:59.358172 1172 tablet_bootstrap.cc:492] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:59.346251 898 tablet_service.cc:1511] Processing CreateTablet for tablet 3462dcc6391741e59dcb5cd08527e50f (DEFAULT_TABLE table=default.Table_13e8ecc0f18d4b909d161b6b56661449 [id=509c7b9d25aa48af9dde3825cfe313d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.360802 758 tablet_service.cc:1511] Processing CreateTablet for tablet a9b7c6e9463342a5a73af12da4447f89 (DEFAULT_TABLE table=default.Table_0393e87d0a314ee898b11935e2f6666d [id=ab93bfbabbad4fad893f7a6dd595540f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.361068 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 with delete_type TABLET_DATA_DELETED (replaced by tablet d5083f98e8e14ccf99a19cdf2bc59018) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.361671 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9b7c6e9463342a5a73af12da4447f89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.362910 882 tablet_service.cc:1558] Processing DeleteTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 with delete_type TABLET_DATA_DELETED (replaced by tablet d5083f98e8e14ccf99a19cdf2bc59018) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.363498 882 tablet_service.cc:1558] Processing DeleteTablet for tablet 1852f7f4ff594639908b03730b68fe86 with delete_type TABLET_DATA_DELETED (replaced by tablet aab3850799254dda9716fd5bf9404159) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.364015 882 tablet_service.cc:1511] Processing CreateTablet for tablet a9b7c6e9463342a5a73af12da4447f89 (DEFAULT_TABLE table=default.Table_0393e87d0a314ee898b11935e2f6666d [id=ab93bfbabbad4fad893f7a6dd595540f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.364470 883 tablet_service.cc:1558] Processing DeleteTablet for tablet 02412333633a4cdebe95a144fa6c363c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.365069 883 tablet_service.cc:1511] Processing CreateTablet for tablet 37765950e67d466683551f67a35e9bdb (DEFAULT_TABLE table=default.Table_119772d89d5f4afa8108d78252d81fe6 [id=129e84f069a74fe9b8ee67eaeba930a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.367046 1176 tablet_bootstrap.cc:492] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:59.367230 1176 ts_tablet_manager.cc:1403] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.105s user 0.002s sys 0.001s
I20260505 07:53:59.368523 1176 raft_consensus.cc:359] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.368748 1176 raft_consensus.cc:385] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.368830 1176 raft_consensus.cc:740] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.369001 744 tablet_service.cc:1511] Processing CreateTablet for tablet 3075e87e078f4d2aa1d329d18902354b (DEFAULT_TABLE table=default.Table_c1cf7717be07400aacdab563b4255450 [id=4d79fded98df46cc884cabdc3fba5d22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.369984 1176 consensus_queue.cc:260] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.370230 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3075e87e078f4d2aa1d329d18902354b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.372442 887 tablet_service.cc:1511] Processing CreateTablet for tablet 3075e87e078f4d2aa1d329d18902354b (DEFAULT_TABLE table=default.Table_c1cf7717be07400aacdab563b4255450 [id=4d79fded98df46cc884cabdc3fba5d22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.372555 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 8b406c4c6ba741bc85eef54c6f576459
I20260505 07:53:59.374461 600 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet a7d8265c05b74b28959d0de41fad1fde
I20260505 07:53:59.376806 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 3075e87e078f4d2aa1d329d18902354b
I20260505 07:53:59.377770 893 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.378413 893 tablet_service.cc:1558] Processing DeleteTablet for tablet a7d8265c05b74b28959d0de41fad1fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.378885 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 3075e87e078f4d2aa1d329d18902354b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.379045 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 32c008bd99224a7595cecc607e5496a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a9494c3b6a99487596a030856b371b68) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.379091 893 tablet_service.cc:1558] Processing DeleteTablet for tablet 3075e87e078f4d2aa1d329d18902354b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.382467 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3462dcc6391741e59dcb5cd08527e50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.383710 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3075e87e078f4d2aa1d329d18902354b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.385736 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02e06d8815034d91986cef12964860ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.387784 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9b7c6e9463342a5a73af12da4447f89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.389082 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 37765950e67d466683551f67a35e9bdb (DEFAULT_TABLE table=default.Table_119772d89d5f4afa8108d78252d81fe6 [id=129e84f069a74fe9b8ee67eaeba930a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.389465 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37765950e67d466683551f67a35e9bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.392364 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37765950e67d466683551f67a35e9bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.392613 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_589d610a49c248009e067d71c961eef9" } new_table_name: "default.Table_8ecdd63342f547ab8e760480eca79fb1" modify_external_catalogs: true
I20260505 07:53:59.395885 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.396970 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet a7d8265c05b74b28959d0de41fad1fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.397560 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 3075e87e078f4d2aa1d329d18902354b (DEFAULT_TABLE table=default.Table_c1cf7717be07400aacdab563b4255450 [id=4d79fded98df46cc884cabdc3fba5d22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.397921 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3075e87e078f4d2aa1d329d18902354b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:59.398821 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1001dd888ecd4166ba217506739f71e8 consensus state has no leader
I20260505 07:53:59.400337 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3462dcc6391741e59dcb5cd08527e50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.403891 1017 tablet_service.cc:1511] Processing CreateTablet for tablet f842d8ac6b2142ffa69945c6087be34b (DEFAULT_TABLE table=default.Table_fa7a8dd4d4e8453180804e6aa56d8ca4 [id=75f94c485caa4e4394a34f1399c9769d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.404206 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f842d8ac6b2142ffa69945c6087be34b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.405992 1172 tablet_bootstrap.cc:654] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.410240 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 423c22891bed462d89e9ec51a079f0f5 (DEFAULT_TABLE table=default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.410367 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 66be7b6c33a342d6890e459d90d40583 (DEFAULT_TABLE table=default.Table_28c935edd66e47959a2df29d39dfe13c [id=fe9a0701ecd44cd28915885958b3b3c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.410332 1025 tablet_service.cc:1511] Processing CreateTablet for tablet e6a5f823db6143319d743d363a77ea47 (DEFAULT_TABLE table=default.Table_359f92a4ff554e3e80468261b0e86966 [id=9930805fdf364f25aea11f859b50bc71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.411068 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 423c22891bed462d89e9ec51a079f0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.411253 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66be7b6c33a342d6890e459d90d40583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.412812 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6a5f823db6143319d743d363a77ea47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.418964 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 080b2984fcf2478a8def6540da63554d (DEFAULT_TABLE table=default.Table_575ab5b128ee4a1baebbe1f90baf005a [id=cdb574c7a4e14a5ea7f3c8b036a8c336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.419302 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080b2984fcf2478a8def6540da63554d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.429710 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 397d33e908af4a0bb8dde26d273be01a from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:59.431082 1027 tablet_service.cc:1511] Processing CreateTablet for tablet a9b7c6e9463342a5a73af12da4447f89 (DEFAULT_TABLE table=default.Table_0393e87d0a314ee898b11935e2f6666d [id=ab93bfbabbad4fad893f7a6dd595540f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.431540 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9b7c6e9463342a5a73af12da4447f89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.435460 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_9f8b49c83f7843318fd7a4645786e1d4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.435858 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 18e909ee93a24ea9842e95357c2ba858 (DEFAULT_TABLE table=default.Table_ed8e2add9bbf436f812de7e3fe2ed692 [id=2a512c2695b540c2a9e14616ee257125]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:59.436008 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f8b49c83f7843318fd7a4645786e1d4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.436167 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18e909ee93a24ea9842e95357c2ba858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.438467 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7b4336be586a4fc1a8dada3651641fd9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.438951 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b4336be586a4fc1a8dada3651641fd9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.439443 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 with delete_type TABLET_DATA_DELETED (replaced by tablet d5083f98e8e14ccf99a19cdf2bc59018) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.439692 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 3075e87e078f4d2aa1d329d18902354b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.440034 1025 tablet_service.cc:1511] Processing CreateTablet for tablet ed0d43d85fcd4d05abf10d415e8dafbe (DEFAULT_TABLE table=default.Table_d2caf74ce2d94a2ea94685efc828218b [id=7ce7150d6a004dcb94857240940133de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.440261 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_09814f6166bb4e6e9978387328078099" } new_table_name: "default.Table_9809ca8cef5f449c851dba1b86ebb1c7" modify_external_catalogs: true
I20260505 07:53:59.440349 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0d43d85fcd4d05abf10d415e8dafbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.441350 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4d333714e70b4e53b695681972d14faf" } new_table_name: "default.Table_e2720de78a7742d5829d7db2ef803f0d" modify_external_catalogs: true
I20260505 07:53:59.442808 1171 ts_tablet_manager.cc:1929] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.442941 1171 log.cc:1199] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/872362aecf6a44b59e586b1fa7637c8b
I20260505 07:53:59.442873 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f580e013bc5c4e53ad7d06fd6d1510eb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.443292 1171 ts_tablet_manager.cc:1950] T 872362aecf6a44b59e586b1fa7637c8b P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
W20260505 07:53:59.443288 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f580e013bc5c4e53ad7d06fd6d1510eb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.446237 599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_fe563815e7df4183a9bfe81907c3c33c" } modify_external_catalogs: true
I20260505 07:53:59.446378 599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_fe563815e7df4183a9bfe81907c3c33c" } modify_external_catalogs: true
I20260505 07:53:59.349786 884 tablet_service.cc:1511] Processing CreateTablet for tablet b6607b7428044e428af9d765c9c9b3ba (DEFAULT_TABLE table=default.Table_6f171a1e79d24e57acfe3aef43b24c7e [id=35a56d340665450ab03ba49b845ae78a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.448570 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6607b7428044e428af9d765c9c9b3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.455807 754 tablet_service.cc:1511] Processing CreateTablet for tablet 37765950e67d466683551f67a35e9bdb (DEFAULT_TABLE table=default.Table_119772d89d5f4afa8108d78252d81fe6 [id=129e84f069a74fe9b8ee67eaeba930a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.455955 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.456282 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37765950e67d466683551f67a35e9bdb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.456681 893 tablet_service.cc:1511] Processing CreateTablet for tablet 423c22891bed462d89e9ec51a079f0f5 (DEFAULT_TABLE table=default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.457039 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 423c22891bed462d89e9ec51a079f0f5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:59.458032 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b5a0cbf9c3d74cea92920189da332c1c consensus state has no leader
W20260505 07:53:59.458482 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 90a13ca612f241fc97ea12124677a0b1 consensus state has no leader
I20260505 07:53:59.465262 760 tablet_service.cc:1511] Processing CreateTablet for tablet f842d8ac6b2142ffa69945c6087be34b (DEFAULT_TABLE table=default.Table_fa7a8dd4d4e8453180804e6aa56d8ca4 [id=75f94c485caa4e4394a34f1399c9769d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.465731 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f842d8ac6b2142ffa69945c6087be34b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.472338 897 tablet_service.cc:1511] Processing CreateTablet for tablet 66be7b6c33a342d6890e459d90d40583 (DEFAULT_TABLE table=default.Table_28c935edd66e47959a2df29d39dfe13c [id=fe9a0701ecd44cd28915885958b3b3c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.472709 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66be7b6c33a342d6890e459d90d40583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.474702 762 tablet_service.cc:1511] Processing CreateTablet for tablet 423c22891bed462d89e9ec51a079f0f5 (DEFAULT_TABLE table=default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.475073 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 423c22891bed462d89e9ec51a079f0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.476073 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 429ccdb56f0b456483cf1debb5de0beb (DEFAULT_TABLE table=default.Table_a44f4d613a6e47e38e9741252ef6956a [id=1a0d30f064af4c2f8e082d5742ba6d28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.476497 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429ccdb56f0b456483cf1debb5de0beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.479612 901 tablet_service.cc:1511] Processing CreateTablet for tablet 080b2984fcf2478a8def6540da63554d (DEFAULT_TABLE table=default.Table_575ab5b128ee4a1baebbe1f90baf005a [id=cdb574c7a4e14a5ea7f3c8b036a8c336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.479662 755 tablet_service.cc:1511] Processing CreateTablet for tablet e6a5f823db6143319d743d363a77ea47 (DEFAULT_TABLE table=default.Table_359f92a4ff554e3e80468261b0e86966 [id=9930805fdf364f25aea11f859b50bc71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.479967 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6a5f823db6143319d743d363a77ea47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.480931 895 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 (DEFAULT_TABLE table=default.Table_fe563815e7df4183a9bfe81907c3c33c [id=fa1b1c24909247538d57d1a37ea9513b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.481112 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 872362aecf6a44b59e586b1fa7637c8b (table default.Table_4d44519a55e0425d8016d1c0c28ed7f7 [id=4a43764f58e442ad892621ae1f29b214]) successfully deleted
I20260505 07:53:59.481526 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6c7c80f7fea34238b6cf32a173f304b5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.482074 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6c7c80f7fea34238b6cf32a173f304b5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.482951 746 tablet_service.cc:1511] Processing CreateTablet for tablet 080b2984fcf2478a8def6540da63554d (DEFAULT_TABLE table=default.Table_575ab5b128ee4a1baebbe1f90baf005a [id=cdb574c7a4e14a5ea7f3c8b036a8c336]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.483263 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080b2984fcf2478a8def6540da63554d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.483263 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_eb1e569ff0a74997b956acbad82f6c49"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.483705 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eb1e569ff0a74997b956acbad82f6c49 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.484781 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_92bd525516eb48ff97bb8fc8c1c2b702" } new_table_name: "default.Table_a2ff5add35c641e9b5341948a1b297c8" modify_external_catalogs: true
I20260505 07:53:59.484902 757 tablet_service.cc:1511] Processing CreateTablet for tablet 429ccdb56f0b456483cf1debb5de0beb (DEFAULT_TABLE table=default.Table_a44f4d613a6e47e38e9741252ef6956a [id=1a0d30f064af4c2f8e082d5742ba6d28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.485059 1178 ts_tablet_manager.cc:1929] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.485179 1178 log.cc:1199] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/cf95f4331e4540ae84e329cf577427b8
I20260505 07:53:59.485204 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429ccdb56f0b456483cf1debb5de0beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.485543 1178 ts_tablet_manager.cc:1950] T cf95f4331e4540ae84e329cf577427b8 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:59.486194 752 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 (DEFAULT_TABLE table=default.Table_fe563815e7df4183a9bfe81907c3c33c [id=fa1b1c24909247538d57d1a37ea9513b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.486466 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3e076892234c1494b1a5c13cbfd2e2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:59.487427 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 02412333633a4cdebe95a144fa6c363c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02412333633a4cdebe95a144fa6c363c already in progress: creating tablet
I20260505 07:53:59.488091 749 tablet_service.cc:1511] Processing CreateTablet for tablet ed0d43d85fcd4d05abf10d415e8dafbe (DEFAULT_TABLE table=default.Table_d2caf74ce2d94a2ea94685efc828218b [id=7ce7150d6a004dcb94857240940133de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.488358 1181 tablet_bootstrap.cc:492] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:59.488521 1181 ts_tablet_manager.cc:1403] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.190s user 0.003s sys 0.000s
W20260505 07:53:59.488556 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 067c32ecccb44bca8bf9cc3f6c229326 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 067c32ecccb44bca8bf9cc3f6c229326 already in progress: creating tablet
W20260505 07:53:59.488751 585 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 3075e87e078f4d2aa1d329d18902354b because the tablet was not found. No further retry: Not found: Tablet not found: 3075e87e078f4d2aa1d329d18902354b
W20260505 07:53:59.489014 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 32c008bd99224a7595cecc607e5496a9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 32c008bd99224a7595cecc607e5496a9 already in progress: creating tablet
W20260505 07:53:59.489217 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 8b406c4c6ba741bc85eef54c6f576459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b406c4c6ba741bc85eef54c6f576459 already in progress: creating tablet
I20260505 07:53:59.489257 1181 raft_consensus.cc:359] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.489444 1181 raft_consensus.cc:385] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.489529 1181 raft_consensus.cc:740] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.491761 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0d43d85fcd4d05abf10d415e8dafbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.491884 882 tablet_service.cc:1511] Processing CreateTablet for tablet f842d8ac6b2142ffa69945c6087be34b (DEFAULT_TABLE table=default.Table_fa7a8dd4d4e8453180804e6aa56d8ca4 [id=75f94c485caa4e4394a34f1399c9769d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.492432 761 tablet_service.cc:1511] Processing CreateTablet for tablet 66be7b6c33a342d6890e459d90d40583 (DEFAULT_TABLE table=default.Table_28c935edd66e47959a2df29d39dfe13c [id=fe9a0701ecd44cd28915885958b3b3c6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:59.492337 425 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 67303.7us
I20260505 07:53:59.493889 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_21673305a26a4342a0be11d98e875447"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.494351 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21673305a26a4342a0be11d98e875447 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.494478 1181 consensus_queue.cc:260] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.499145 1181 ts_tablet_manager.cc:1434] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260505 07:53:59.499447 1181 tablet_bootstrap.cc:492] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:59.499907 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66be7b6c33a342d6890e459d90d40583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.501037 1181 tablet_bootstrap.cc:654] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.501318 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 46274a44a06b4306b5dac43c66c85a80 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:59.508561 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 252) took 1861 ms. Trace:
I20260505 07:53:59.511803 1178 rpcz_store.cc:276] 0505 07:53:57.647546 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.647616 (+ 70us) service_pool.cc:224] Handling call
0505 07:53:59.329855 (+1682239us) ts_tablet_manager.cc:1151] Deleting tablet cf95f4331e4540ae84e329cf577427b8
0505 07:53:59.329865 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:59.344064 (+ 14199us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:59.484978 (+140914us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:59.508523 (+ 23545us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1681398}
I20260505 07:53:59.512280 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet cf95f4331e4540ae84e329cf577427b8 (table default.Table_8a35dff4878b4429b4a6600b8259fd83 [id=0e331c73bbf842dcac6c971a6688f58b]) successfully deleted
I20260505 07:53:59.508658 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 25a74b65d5d6459293ab4461f4d34cf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.512930 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 253) took 1864 ms. Trace:
I20260505 07:53:59.513103 1178 rpcz_store.cc:276] 0505 07:53:57.648109 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.648165 (+ 56us) service_pool.cc:224] Handling call
0505 07:53:59.512795 (+1864630us) ts_tablet_manager.cc:1151] Deleting tablet 202a237ca580482c98803683a77d96d2
0505 07:53:59.512804 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:59.512899 (+ 95us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1864463}
W20260505 07:53:59.513654 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 202a237ca580482c98803683a77d96d2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 202a237ca580482c98803683a77d96d2 already in progress: creating tablet
I20260505 07:53:59.514362 758 tablet_service.cc:1558] Processing DeleteTablet for tablet a7d8265c05b74b28959d0de41fad1fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.514254 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_5f0d98b4110b4596a6de3749d10302c9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.515117 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet a7d8265c05b74b28959d0de41fad1fde because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7d8265c05b74b28959d0de41fad1fde already in progress: creating tablet
W20260505 07:53:59.515127 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f0d98b4110b4596a6de3749d10302c9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.518355 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 254) took 1869 ms. Trace:
I20260505 07:53:59.519336 1178 rpcz_store.cc:276] 0505 07:53:57.648559 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.648599 (+ 40us) service_pool.cc:224] Handling call
0505 07:53:59.517580 (+1868981us) ts_tablet_manager.cc:1151] Deleting tablet 23fa401bb4634022bca3c6b23421e4d5
0505 07:53:59.517591 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:59.518156 (+ 565us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1868627}
W20260505 07:53:59.520887 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 23fa401bb4634022bca3c6b23421e4d5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 23fa401bb4634022bca3c6b23421e4d5 already in progress: creating tablet
I20260505 07:53:59.522598 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080b2984fcf2478a8def6540da63554d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.522794 897 tablet_service.cc:1511] Processing CreateTablet for tablet ed0d43d85fcd4d05abf10d415e8dafbe (DEFAULT_TABLE table=default.Table_d2caf74ce2d94a2ea94685efc828218b [id=7ce7150d6a004dcb94857240940133de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.523095 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3e076892234c1494b1a5c13cbfd2e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.522754 890 tablet_service.cc:1511] Processing CreateTablet for tablet 429ccdb56f0b456483cf1debb5de0beb (DEFAULT_TABLE table=default.Table_a44f4d613a6e47e38e9741252ef6956a [id=1a0d30f064af4c2f8e082d5742ba6d28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.523340 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed0d43d85fcd4d05abf10d415e8dafbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.523617 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 429ccdb56f0b456483cf1debb5de0beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.523643 893 tablet_service.cc:1511] Processing CreateTablet for tablet e6a5f823db6143319d743d363a77ea47 (DEFAULT_TABLE table=default.Table_359f92a4ff554e3e80468261b0e86966 [id=9930805fdf364f25aea11f859b50bc71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.523998 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6a5f823db6143319d743d363a77ea47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.527933 1544 raft_consensus.cc:493] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:59.528074 1544 raft_consensus.cc:515] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.528798 1544 leader_election.cc:290] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:53:59.529564 991 leader_election.cc:343] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: e1a9420a81844d2092381123d9297661
I20260505 07:53:59.533625 1181 tablet_bootstrap.cc:492] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:59.533842 1181 ts_tablet_manager.cc:1403] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.003s
I20260505 07:53:59.534597 1181 raft_consensus.cc:359] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.534828 1181 raft_consensus.cc:385] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.534942 1181 raft_consensus.cc:740] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.535176 1181 consensus_queue.cc:260] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:53:59.535179 988 leader_election.cc:343] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: e1a9420a81844d2092381123d9297661
I20260505 07:53:59.535352 887 tablet_service.cc:1511] Processing CreateTablet for tablet 18e909ee93a24ea9842e95357c2ba858 (DEFAULT_TABLE table=default.Table_ed8e2add9bbf436f812de7e3fe2ed692 [id=2a512c2695b540c2a9e14616ee257125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.535490 988 leader_election.cc:304] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:59.535706 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18e909ee93a24ea9842e95357c2ba858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.535754 1544 raft_consensus.cc:2749] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:59.536103 1181 ts_tablet_manager.cc:1434] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:53:59.543637 1172 tablet_bootstrap.cc:492] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:59.543793 1172 ts_tablet_manager.cc:1403] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.186s user 0.003s sys 0.000s
I20260505 07:53:59.544595 1172 raft_consensus.cc:359] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.544862 1172 raft_consensus.cc:385] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.545014 1172 raft_consensus.cc:740] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.545282 1172 consensus_queue.cc:260] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.545470 751 tablet_service.cc:1511] Processing CreateTablet for tablet 18e909ee93a24ea9842e95357c2ba858 (DEFAULT_TABLE table=default.Table_ed8e2add9bbf436f812de7e3fe2ed692 [id=2a512c2695b540c2a9e14616ee257125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.546676 1172 ts_tablet_manager.cc:1434] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:59.547142 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f842d8ac6b2142ffa69945c6087be34b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.547479 1171 tablet_replica.cc:333] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:59.548960 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 (DEFAULT_TABLE table=default.Table_fe563815e7df4183a9bfe81907c3c33c [id=fa1b1c24909247538d57d1a37ea9513b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.549911 1171 raft_consensus.cc:2243] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.550140 1171 raft_consensus.cc:2272] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.550709 1171 ts_tablet_manager.cc:1916] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.551818 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_b00ce8f351ce43edb2431c6e621d329e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.552381 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b00ce8f351ce43edb2431c6e621d329e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.553607 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_3c9fe3aa59174b11a7f5f4488741deec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.554150 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3c9fe3aa59174b11a7f5f4488741deec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:53:59.558116 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d5083f98e8e14ccf99a19cdf2bc59018 consensus state has no leader
I20260505 07:53:59.560473 1172 tablet_bootstrap.cc:492] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:59.561126 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 4f3e076892234c1494b1a5c13cbfd2e2
I20260505 07:53:59.562503 1028 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.562901 1172 tablet_bootstrap.cc:654] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.566926 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18e909ee93a24ea9842e95357c2ba858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.573277 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_01a5eabcd2ff41fca0d5c94e26775754" } modify_external_catalogs: true
I20260505 07:53:59.573447 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_01a5eabcd2ff41fca0d5c94e26775754" } modify_external_catalogs: true
I20260505 07:53:59.574719 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6b9d99bdf8624bba90ba61af22f36dc2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.577670 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b9d99bdf8624bba90ba61af22f36dc2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.578598 601 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_3a76cf09eaed4c9a8c267714a164dd66" } modify_external_catalogs: true
I20260505 07:53:59.578743 601 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_3a76cf09eaed4c9a8c267714a164dd66" } modify_external_catalogs: true
I20260505 07:53:59.578941 598 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_563025047e4142d4807295cff836ea99" } modify_external_catalogs: true
I20260505 07:53:59.579154 598 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_563025047e4142d4807295cff836ea99" } modify_external_catalogs: true
I20260505 07:53:59.580428 599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0555c54d8e394cb1a8509950422c9ac8" } modify_external_catalogs: true
I20260505 07:53:59.580592 599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0555c54d8e394cb1a8509950422c9ac8" } modify_external_catalogs: true
I20260505 07:53:59.565937 901 tablet_service.cc:1558] Processing DeleteTablet for tablet ccdde3e5a13e436d9f2328831e155adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.568370 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.522679 1178 tablet_replica.cc:333] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:59.589618 1178 raft_consensus.cc:2243] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.589895 1178 raft_consensus.cc:2272] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.590554 1178 ts_tablet_manager.cc:1916] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.591002 1181 tablet_bootstrap.cc:492] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:59.592734 1181 tablet_bootstrap.cc:654] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.595469 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f3e076892234c1494b1a5c13cbfd2e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.596604 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f580e013bc5c4e53ad7d06fd6d1510eb" } new_table_name: "default.Table_ee5371f299e44106859f3663fe6052b7" modify_external_catalogs: true
I20260505 07:53:59.601316 890 tablet_service.cc:1511] Processing CreateTablet for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (DEFAULT_TABLE table=default.Table_1f4b1af520d64a108ff4289226a1561d [id=a238daba389c412ea67318d99cfb5559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.601861 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.603803 1032 tablet_service.cc:1511] Processing CreateTablet for tablet df3a49b59ed6485f970074391f932482 (DEFAULT_TABLE table=default.Table_7e8b9279f3e244a99b8ba4a25e7c569f [id=306149ae47a745edabf863d5004d5012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.604288 1030 tablet_service.cc:1511] Processing CreateTablet for tablet ec145bdc0dab4642ba233bd0da2c436f (DEFAULT_TABLE table=default.Table_9201abe14fd54f8880602582b1f74cc8 [id=2f4387e6af20481fa79a3b8849a529cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.606240 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df3a49b59ed6485f970074391f932482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.608088 1031 tablet_service.cc:1511] Processing CreateTablet for tablet d77ff90ec2ff4b6eb1795d149459c23a (DEFAULT_TABLE table=default.Table_2f8e689cd77249e9a31aa6d90f55b4ae [id=f608e93cbc64485297c6dc2609a6eb87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.608451 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d77ff90ec2ff4b6eb1795d149459c23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.608592 901 tablet_service.cc:1511] Processing CreateTablet for tablet ec145bdc0dab4642ba233bd0da2c436f (DEFAULT_TABLE table=default.Table_9201abe14fd54f8880602582b1f74cc8 [id=2f4387e6af20481fa79a3b8849a529cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.608686 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec145bdc0dab4642ba233bd0da2c436f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.608963 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec145bdc0dab4642ba233bd0da2c436f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.611940 897 tablet_service.cc:1511] Processing CreateTablet for tablet df3a49b59ed6485f970074391f932482 (DEFAULT_TABLE table=default.Table_7e8b9279f3e244a99b8ba4a25e7c569f [id=306149ae47a745edabf863d5004d5012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.612311 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df3a49b59ed6485f970074391f932482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.616871 893 tablet_service.cc:1511] Processing CreateTablet for tablet 27bacc97d7534284970f218cf68ac75b (DEFAULT_TABLE table=default.Table_db40b13ba59549c6bae350382395bceb [id=db770342dce84ba1ab28d029ddb65d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.617251 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27bacc97d7534284970f218cf68ac75b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.621454 593 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_13e8ecc0f18d4b909d161b6b56661449" } new_table_name: "default.Table_6de81e84d3e44828a5b8026b551bdeb3" modify_external_catalogs: true
I20260505 07:53:59.624863 756 tablet_service.cc:1511] Processing CreateTablet for tablet d77ff90ec2ff4b6eb1795d149459c23a (DEFAULT_TABLE table=default.Table_2f8e689cd77249e9a31aa6d90f55b4ae [id=f608e93cbc64485297c6dc2609a6eb87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.625330 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d77ff90ec2ff4b6eb1795d149459c23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.629302 1544 raft_consensus.cc:493] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:59.629469 1544 raft_consensus.cc:515] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.630110 1544 leader_election.cc:290] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:59.631035 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (DEFAULT_TABLE table=default.Table_1f4b1af520d64a108ff4289226a1561d [id=a238daba389c412ea67318d99cfb5559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.631436 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:59.631768 991 leader_election.cc:343] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8573f8bd424641438e2aa6ff6869180e
I20260505 07:53:59.635421 600 master_service.cc:954] ReplaceTablet: received request to replace tablet 472441bd29704be59aabb0f7a4964bb0 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:59.635959 1176 ts_tablet_manager.cc:1434] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.103s user 0.003s sys 0.000s
W20260505 07:53:59.638837 988 leader_election.cc:343] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 8573f8bd424641438e2aa6ff6869180e
I20260505 07:53:59.638983 988 leader_election.cc:304] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:59.639227 1544 raft_consensus.cc:2749] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:59.639360 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.644796 758 tablet_service.cc:1511] Processing CreateTablet for tablet df3a49b59ed6485f970074391f932482 (DEFAULT_TABLE table=default.Table_7e8b9279f3e244a99b8ba4a25e7c569f [id=306149ae47a745edabf863d5004d5012]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.645287 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df3a49b59ed6485f970074391f932482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.647538 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 0459f2d45a2d44f1a73e0a0fff53866c
I20260505 07:53:59.648991 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.649237 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 02716f62aa354d8f839266661b7f05ea (DEFAULT_TABLE table=default.Table_a86abb495238445381ac0202804b4ca0 [id=4d3b183ab74a4078beb7583c9e60e2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.649361 882 tablet_service.cc:1511] Processing CreateTablet for tablet 02716f62aa354d8f839266661b7f05ea (DEFAULT_TABLE table=default.Table_a86abb495238445381ac0202804b4ca0 [id=4d3b183ab74a4078beb7583c9e60e2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.649583 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02716f62aa354d8f839266661b7f05ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.649756 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02716f62aa354d8f839266661b7f05ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.651736 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.653290 1172 tablet_bootstrap.cc:492] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:59.653510 1172 ts_tablet_manager.cc:1403] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.093s user 0.002s sys 0.002s
I20260505 07:53:59.654630 1172 raft_consensus.cc:359] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.654945 1172 raft_consensus.cc:385] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.655058 1172 raft_consensus.cc:740] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.655328 1172 consensus_queue.cc:260] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.655937 1172 ts_tablet_manager.cc:1434] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:53:59.661149 1172 tablet_bootstrap.cc:492] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:59.663041 1172 tablet_bootstrap.cc:654] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.667706 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (DEFAULT_TABLE table=default.Table_c6c7a2a777c24a65a8217641ac14941c [id=eddd2f477fd142228488291699aa5e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.668183 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9963a0f1a54ee9a73486bb1801ee74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.668869 761 tablet_service.cc:1511] Processing CreateTablet for tablet 27bacc97d7534284970f218cf68ac75b (DEFAULT_TABLE table=default.Table_db40b13ba59549c6bae350382395bceb [id=db770342dce84ba1ab28d029ddb65d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.669369 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27bacc97d7534284970f218cf68ac75b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.672780 601 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet f121fdd001ec4ea38ccd24a6afd9e51c
I20260505 07:53:59.673592 598 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 3d83e9d4243e409794ffe8a342bce6b3
I20260505 07:53:59.673751 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.674245 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 4783dd0b2fbe40219852d850fc0d9a82
I20260505 07:53:59.674266 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d83e9d4243e409794ffe8a342bce6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.674790 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.673668 754 tablet_service.cc:1511] Processing CreateTablet for tablet ec145bdc0dab4642ba233bd0da2c436f (DEFAULT_TABLE table=default.Table_9201abe14fd54f8880602582b1f74cc8 [id=2f4387e6af20481fa79a3b8849a529cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.678992 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec145bdc0dab4642ba233bd0da2c436f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.679768 760 tablet_service.cc:1511] Processing CreateTablet for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (DEFAULT_TABLE table=default.Table_1f4b1af520d64a108ff4289226a1561d [id=a238daba389c412ea67318d99cfb5559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.680141 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.681202 1176 tablet_bootstrap.cc:492] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:59.683007 1176 tablet_bootstrap.cc:654] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.665118 1171 ts_tablet_manager.cc:1929] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.667317 890 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (DEFAULT_TABLE table=default.Table_c6c7a2a777c24a65a8217641ac14941c [id=eddd2f477fd142228488291699aa5e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.675500 886 tablet_service.cc:1558] Processing DeleteTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.675699 884 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d83e9d4243e409794ffe8a342bce6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.687245 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9963a0f1a54ee9a73486bb1801ee74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.690721 1366 raft_consensus.cc:493] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:59.690855 1366 raft_consensus.cc:515] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.691431 1366 leader_election.cc:290] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:59.692078 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1852f7f4ff594639908b03730b68fe86" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:53:59.692526 854 leader_election.cc:343] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:59.692802 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1852f7f4ff594639908b03730b68fe86" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:53:59.693325 856 leader_election.cc:343] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:59.693482 856 leader_election.cc:304] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:59.693747 1366 raft_consensus.cc:2749] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:53:59.694165 585 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 58646us
I20260505 07:53:59.695722 749 tablet_service.cc:1511] Processing CreateTablet for tablet 02716f62aa354d8f839266661b7f05ea (DEFAULT_TABLE table=default.Table_a86abb495238445381ac0202804b4ca0 [id=4d3b183ab74a4078beb7583c9e60e2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.696393 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02716f62aa354d8f839266661b7f05ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.693037 895 tablet_service.cc:1511] Processing CreateTablet for tablet d77ff90ec2ff4b6eb1795d149459c23a (DEFAULT_TABLE table=default.Table_2f8e689cd77249e9a31aa6d90f55b4ae [id=f608e93cbc64485297c6dc2609a6eb87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.701206 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d77ff90ec2ff4b6eb1795d149459c23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.703745 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet e85b44337f62494c997c0008fc492aab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.704167 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 27bacc97d7534284970f218cf68ac75b (DEFAULT_TABLE table=default.Table_db40b13ba59549c6bae350382395bceb [id=db770342dce84ba1ab28d029ddb65d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.704634 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27bacc97d7534284970f218cf68ac75b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.706637 884 tablet_service.cc:1511] Processing CreateTablet for tablet 09b46ff0101a4e529077b9e30492ace7 (DEFAULT_TABLE table=default.Table_c12be1928760438fab7c135df431bc79 [id=d567651314004cf4b33a8dfc1ef30698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.707005 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09b46ff0101a4e529077b9e30492ace7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.708263 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 09b46ff0101a4e529077b9e30492ace7 (DEFAULT_TABLE table=default.Table_c12be1928760438fab7c135df431bc79 [id=d567651314004cf4b33a8dfc1ef30698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.708806 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09b46ff0101a4e529077b9e30492ace7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.712531 1181 tablet_bootstrap.cc:492] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:59.712791 1181 ts_tablet_manager.cc:1403] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.122s user 0.005s sys 0.004s
I20260505 07:53:59.713722 1181 raft_consensus.cc:359] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.714047 1181 raft_consensus.cc:385] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.714184 1181 raft_consensus.cc:740] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.714550 1181 consensus_queue.cc:260] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.714864 886 tablet_service.cc:1511] Processing CreateTablet for tablet 4ef22101fe014e21adea6e10f4b550e7 (DEFAULT_TABLE table=default.Table_89965306827247e99ff9f34d13e63c54 [id=567b03439f3a412aa6c1a4f4bac48f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.715201 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ef22101fe014e21adea6e10f4b550e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.717051 1181 ts_tablet_manager.cc:1434] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:53:59.717437 1181 tablet_bootstrap.cc:492] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:59.722716 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.724207 755 tablet_service.cc:1511] Processing CreateTablet for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (DEFAULT_TABLE table=default.Table_c6c7a2a777c24a65a8217641ac14941c [id=eddd2f477fd142228488291699aa5e4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.724608 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b9963a0f1a54ee9a73486bb1801ee74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.724653 901 tablet_service.cc:1511] Processing CreateTablet for tablet c3be796a788e4465a36bb5be44918be1 (DEFAULT_TABLE table=default.Table_37add05ae60c4b40b655ffe723848dfd [id=3a9a2fa1381b40608a07038149eae25e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:53:59.724889 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 60772a9697ba417b879dc6025eeb8eec has no consensus state
I20260505 07:53:59.724956 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3be796a788e4465a36bb5be44918be1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:59.728749 593 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3462dcc6391741e59dcb5cd08527e50f consensus state has no leader
I20260505 07:53:59.729377 1178 ts_tablet_manager.cc:1929] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.729568 1178 log.cc:1199] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/5f57568e1c4c4573ac18d9f621eeb4c7
I20260505 07:53:59.729962 1178 ts_tablet_manager.cc:1950] T 5f57568e1c4c4573ac18d9f621eeb4c7 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:53:59.730373 1544 raft_consensus.cc:493] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:59.730585 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 32c008bd99224a7595cecc607e5496a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a9494c3b6a99487596a030856b371b68) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.730572 1544 raft_consensus.cc:515] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.730782 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 255) took 2081 ms. Trace:
I20260505 07:53:59.730878 1178 rpcz_store.cc:276] 0505 07:53:57.648939 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.649036 (+ 97us) service_pool.cc:224] Handling call
0505 07:53:59.522346 (+1873310us) ts_tablet_manager.cc:1151] Deleting tablet 5f57568e1c4c4573ac18d9f621eeb4c7
0505 07:53:59.522361 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:59.593786 (+ 71425us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:59.729319 (+135533us) tablet_metadata.cc:665] Metadata flushed
0505 07:53:59.730722 (+ 1403us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1872900}
I20260505 07:53:59.731148 1175 tablet_replica.cc:333] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:59.731300 1175 raft_consensus.cc:2243] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.731384 1544 leader_election.cc:290] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:53:59.732437 991 leader_election.cc:343] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: c7652191b1d44ba59d856a2b0d7017b1
I20260505 07:53:59.732851 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_3621941a687641ecbcaa200aaefbe112"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.733273 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 4f3e076892234c1494b1a5c13cbfd2e2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f3e076892234c1494b1a5c13cbfd2e2 already in progress: creating tablet
W20260505 07:53:59.733354 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3621941a687641ecbcaa200aaefbe112 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.733588 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.734053 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.735044 599 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4b3acc65322b4dfe93c4f1d049b573e5" } new_table_name: "default.Table_8091d8a01cbf47f3b270874cd5e6140e" modify_external_catalogs: true
I20260505 07:53:59.735736 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d83e9d4243e409794ffe8a342bce6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.736207 1175 raft_consensus.cc:2272] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:53:59.736516 988 leader_election.cc:343] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: c7652191b1d44ba59d856a2b0d7017b1
I20260505 07:53:59.736635 988 leader_election.cc:304] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:59.737049 1544 raft_consensus.cc:2749] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:53:59.737349 1175 ts_tablet_manager.cc:1916] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.737664 886 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a60c015136246a3b02f70254c0bb67f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.738197 600 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 472441bd29704be59aabb0f7a4964bb0
I20260505 07:53:59.738629 600 catalog_manager.cc:6741] ReplaceTablet: tablet 472441bd29704be59aabb0f7a4964bb0 deleted and replaced by tablet e7887c3462f74740b314e3bec36e280d
I20260505 07:53:59.739049 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 472441bd29704be59aabb0f7a4964bb0 with delete_type TABLET_DATA_DELETED (replaced by tablet e7887c3462f74740b314e3bec36e280d) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.739130 886 tablet_service.cc:1558] Processing DeleteTablet for tablet 472441bd29704be59aabb0f7a4964bb0 with delete_type TABLET_DATA_DELETED (replaced by tablet e7887c3462f74740b314e3bec36e280d) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.739687 1025 tablet_service.cc:1511] Processing CreateTablet for tablet c3be796a788e4465a36bb5be44918be1 (DEFAULT_TABLE table=default.Table_37add05ae60c4b40b655ffe723848dfd [id=3a9a2fa1381b40608a07038149eae25e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.740185 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3be796a788e4465a36bb5be44918be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.686480 1171 log.cc:1199] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/202a237ca580482c98803683a77d96d2
I20260505 07:53:59.675845 887 tablet_service.cc:1558] Processing DeleteTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.744148 1171 ts_tablet_manager.cc:1950] T 202a237ca580482c98803683a77d96d2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:53:59.744760 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 202a237ca580482c98803683a77d96d2 (table default.Table_2f6f14275b8847be96f68a761156a854 [id=9c830b1f855348299c40b40610589233]) successfully deleted
W20260505 07:53:59.745090 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet d7e0c8ad716e443588d4ae3aa28ed4bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d7e0c8ad716e443588d4ae3aa28ed4bf already in progress: creating tablet
W20260505 07:53:59.745462 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 067c32ecccb44bca8bf9cc3f6c229326 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 067c32ecccb44bca8bf9cc3f6c229326 already in progress: creating tablet
I20260505 07:53:59.745594 1171 tablet_replica.cc:333] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:53:59.745815 1171 raft_consensus.cc:2243] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.746097 1171 raft_consensus.cc:2272] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.748824 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7f7ad7a477f249319d0428cd775dc541" } new_table_name: "default.Table_696d4b08028248efafc4a12885184881" modify_external_catalogs: true
I20260505 07:53:59.750032 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_9c83e246b9a74343acdf478231329b39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.750502 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c83e246b9a74343acdf478231329b39 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.753274 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_0410f2bbd0504738ba4aa1f7412e3405"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.753768 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0410f2bbd0504738ba4aa1f7412e3405 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.754796 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f75ef4b5e6ae433f8c6bee72fc30eaf2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:53:59.755230 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f75ef4b5e6ae433f8c6bee72fc30eaf2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.755355 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 4ef22101fe014e21adea6e10f4b550e7 (DEFAULT_TABLE table=default.Table_89965306827247e99ff9f34d13e63c54 [id=567b03439f3a412aa6c1a4f4bac48f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.755774 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ef22101fe014e21adea6e10f4b550e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.756776 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.756985 1176 tablet_bootstrap.cc:492] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:59.757135 1176 ts_tablet_manager.cc:1403] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.120s user 0.003s sys 0.000s
I20260505 07:53:59.757905 1172 tablet_bootstrap.cc:492] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:59.757922 1176 raft_consensus.cc:359] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.758091 1176 raft_consensus.cc:385] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.758176 1176 raft_consensus.cc:740] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.758188 1172 ts_tablet_manager.cc:1403] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.097s user 0.011s sys 0.029s
I20260505 07:53:59.758407 1176 consensus_queue.cc:260] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.758618 1175 ts_tablet_manager.cc:1929] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:53:59.758720 1175 log.cc:1199] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/32c008bd99224a7595cecc607e5496a9
I20260505 07:53:59.759107 1175 ts_tablet_manager.cc:1950] T 32c008bd99224a7595cecc607e5496a9 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:53:59.759477 1172 raft_consensus.cc:359] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.759716 1172 raft_consensus.cc:385] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.760321 1172 raft_consensus.cc:740] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.760613 1172 consensus_queue.cc:260] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.761217 1172 ts_tablet_manager.cc:1434] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:53:59.761641 1172 tablet_bootstrap.cc:492] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:59.763334 1172 tablet_bootstrap.cc:654] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.764000 1181 tablet_bootstrap.cc:654] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.769570 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 4b528d6c36ac4920b6ea0a6e26cca269 (DEFAULT_TABLE table=default.Table_79ed240074b343fabbdd206872328277 [id=7cbbeae00a3d4f85bcaef70b7aaa69de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.769995 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b528d6c36ac4920b6ea0a6e26cca269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.770900 761 tablet_service.cc:1558] Processing DeleteTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.771869 1031 tablet_service.cc:1511] Processing CreateTablet for tablet d5083f98e8e14ccf99a19cdf2bc59018 (DEFAULT_TABLE table=default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.772239 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5083f98e8e14ccf99a19cdf2bc59018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.772352 887 tablet_service.cc:1511] Processing CreateTablet for tablet 2045202714c14aa388d1e8bc18e2063d (DEFAULT_TABLE table=default.Table_657cc0ba6f494479bd433c8c0279c734 [id=f191fbb56e234f10a9f94b2b998781bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.772693 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2045202714c14aa388d1e8bc18e2063d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.773825 886 tablet_service.cc:1511] Processing CreateTablet for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (DEFAULT_TABLE table=default.Table_7d1747ea770445058549830f424a7aa3 [id=438fe94f1aab49919a5aae9ccab8f491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.774276 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f207a3ffbc94b01abd9e9d6837f5c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.775895 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 2045202714c14aa388d1e8bc18e2063d (DEFAULT_TABLE table=default.Table_657cc0ba6f494479bd433c8c0279c734 [id=f191fbb56e234f10a9f94b2b998781bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.776196 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2045202714c14aa388d1e8bc18e2063d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:59.791656 599 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9a06f77255544d7dabb71a4f70ada809 consensus state has no leader
I20260505 07:53:59.794720 1172 tablet_bootstrap.cc:492] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:59.794931 1172 ts_tablet_manager.cc:1403] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.033s user 0.005s sys 0.004s
I20260505 07:53:59.795763 1172 raft_consensus.cc:359] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.796037 1172 raft_consensus.cc:385] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.796183 1172 raft_consensus.cc:740] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.797098 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (DEFAULT_TABLE table=default.Table_7d1747ea770445058549830f424a7aa3 [id=438fe94f1aab49919a5aae9ccab8f491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.797502 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f207a3ffbc94b01abd9e9d6837f5c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.797605 1181 tablet_bootstrap.cc:492] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:59.797724 1181 ts_tablet_manager.cc:1403] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.080s user 0.003s sys 0.000s
I20260505 07:53:59.798735 1181 raft_consensus.cc:359] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:59.799006 1181 raft_consensus.cc:385] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.799758 1181 raft_consensus.cc:740] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.800002 1181 consensus_queue.cc:260] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:59.800619 1181 ts_tablet_manager.cc:1434] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260505 07:53:59.734061 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 0459f2d45a2d44f1a73e0a0fff53866c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0459f2d45a2d44f1a73e0a0fff53866c already in progress: creating tablet
W20260505 07:53:59.805106 585 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 71035.3us
I20260505 07:53:59.805729 1172 consensus_queue.cc:260] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.806388 1172 ts_tablet_manager.cc:1434] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20260505 07:53:59.807054 893 tablet_service.cc:1511] Processing CreateTablet for tablet d5083f98e8e14ccf99a19cdf2bc59018 (DEFAULT_TABLE table=default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.807749 1172 tablet_bootstrap.cc:492] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:53:59.809571 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a7fe9c7199cc4039ae877dc3d4aab17e consensus state has no leader
I20260505 07:53:59.811630 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5083f98e8e14ccf99a19cdf2bc59018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.811928 754 tablet_service.cc:1511] Processing CreateTablet for tablet 09b46ff0101a4e529077b9e30492ace7 (DEFAULT_TABLE table=default.Table_c12be1928760438fab7c135df431bc79 [id=d567651314004cf4b33a8dfc1ef30698]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.812368 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09b46ff0101a4e529077b9e30492ace7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.814410 1181 tablet_bootstrap.cc:492] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:59.815274 1544 raft_consensus.cc:493] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:59.815742 1544 raft_consensus.cc:515] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.816675 1544 leader_election.cc:290] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:53:59.817404 991 leader_election.cc:343] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: e85b44337f62494c997c0008fc492aab
I20260505 07:53:59.817683 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85b44337f62494c997c0008fc492aab" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:59.817886 781 raft_consensus.cc:2468] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:53:59.818744 988 leader_election.cc:304] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:53:59.819028 1544 raft_consensus.cc:2804] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:53:59.819188 1544 raft_consensus.cc:493] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:53:59.819311 1544 raft_consensus.cc:3060] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:53:59.822120 1172 tablet_bootstrap.cc:654] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.822731 1544 raft_consensus.cc:515] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.823359 1544 leader_election.cc:290] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:53:59.827872 761 tablet_service.cc:1511] Processing CreateTablet for tablet c3be796a788e4465a36bb5be44918be1 (DEFAULT_TABLE table=default.Table_37add05ae60c4b40b655ffe723848dfd [id=3a9a2fa1381b40608a07038149eae25e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.828383 1181 tablet_bootstrap.cc:654] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.828943 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3be796a788e4465a36bb5be44918be1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.829413 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85b44337f62494c997c0008fc492aab" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:53:59.829622 781 raft_consensus.cc:3060] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:53:59.831900 991 leader_election.cc:343] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: e85b44337f62494c997c0008fc492aab
I20260505 07:53:59.832959 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 256) took 2183 ms. Trace:
I20260505 07:53:59.833092 1178 rpcz_store.cc:276] 0505 07:53:57.649393 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.666031 (+ 16638us) service_pool.cc:224] Handling call
0505 07:53:59.832793 (+2166762us) ts_tablet_manager.cc:1151] Deleting tablet 62d4bc3154704ba2bfde83bca10f812d
0505 07:53:59.832802 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:59.832903 (+ 101us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2166552}
W20260505 07:53:59.833643 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 62d4bc3154704ba2bfde83bca10f812d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 62d4bc3154704ba2bfde83bca10f812d already in progress: creating tablet
I20260505 07:53:59.833998 1172 tablet_bootstrap.cc:492] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:59.834163 1172 ts_tablet_manager.cc:1403] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.003s
I20260505 07:53:59.834213 749 tablet_service.cc:1558] Processing DeleteTablet for tablet 472441bd29704be59aabb0f7a4964bb0 with delete_type TABLET_DATA_DELETED (replaced by tablet e7887c3462f74740b314e3bec36e280d) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.834323 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 257) took 2167 ms. Trace:
I20260505 07:53:59.834407 1178 rpcz_store.cc:276] 0505 07:53:57.666375 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.666419 (+ 44us) service_pool.cc:224] Handling call
0505 07:53:59.834131 (+2167712us) ts_tablet_manager.cc:1151] Deleting tablet bde59d52fd234343b1ca13a7d4aae6bf
0505 07:53:59.834141 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:59.834273 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2167503}
W20260505 07:53:59.834786 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet bde59d52fd234343b1ca13a7d4aae6bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bde59d52fd234343b1ca13a7d4aae6bf already in progress: creating tablet
I20260505 07:53:59.835161 1178 tablet_replica.cc:333] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:53:59.835035 1172 raft_consensus.cc:359] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.835388 1172 raft_consensus.cc:385] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.835510 1172 raft_consensus.cc:740] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.835521 1178 raft_consensus.cc:2243] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.835762 1172 consensus_queue.cc:260] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.835844 1178 raft_consensus.cc:2272] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.836217 1172 ts_tablet_manager.cc:1434] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:53:59.836530 1178 ts_tablet_manager.cc:1916] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.836571 1172 tablet_bootstrap.cc:492] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:53:59.839340 1172 tablet_bootstrap.cc:654] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.839928 1176 ts_tablet_manager.cc:1434] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.083s user 0.002s sys 0.000s
I20260505 07:53:59.840605 1176 tablet_bootstrap.cc:492] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:59.840840 1181 tablet_bootstrap.cc:492] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:59.841012 1181 ts_tablet_manager.cc:1403] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.003s
I20260505 07:53:59.841751 1181 raft_consensus.cc:359] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:59.841936 1181 raft_consensus.cc:385] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.842022 1181 raft_consensus.cc:740] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.842293 1181 consensus_queue.cc:260] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:53:59.842612 1176 tablet_bootstrap.cc:654] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.842932 1181 ts_tablet_manager.cc:1434] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:53:59.843617 1181 tablet_bootstrap.cc:492] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:59.845213 1181 tablet_bootstrap.cc:654] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.847154 1171 ts_tablet_manager.cc:1916] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:53:59.849136 587 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet a7fe9c7199cc4039ae877dc3d4aab17e (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a7fe9c7199cc4039ae877dc3d4aab17e consensus state has no leader
I20260505 07:53:59.849786 746 tablet_service.cc:1511] Processing CreateTablet for tablet 4b528d6c36ac4920b6ea0a6e26cca269 (DEFAULT_TABLE table=default.Table_79ed240074b343fabbdd206872328277 [id=7cbbeae00a3d4f85bcaef70b7aaa69de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.850193 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b528d6c36ac4920b6ea0a6e26cca269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.851346 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1523b3ae7cd94c599e60a187bc677fc0" } modify_external_catalogs: true
I20260505 07:53:59.851553 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1523b3ae7cd94c599e60a187bc677fc0" } modify_external_catalogs: true
I20260505 07:53:59.851765 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 141665fcd59048bf926acd0287f1853e from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:59.852481 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_fa7a8dd4d4e8453180804e6aa56d8ca4" } new_table_name: "default.Table_0c1bd9ecad69401ab663ea12961bd890" modify_external_catalogs: true
I20260505 07:53:59.852627 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 32c008bd99224a7595cecc607e5496a9 (table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]) successfully deleted
W20260505 07:53:59.852851 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 3d83e9d4243e409794ffe8a342bce6b3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3d83e9d4243e409794ffe8a342bce6b3 already in progress: creating tablet
W20260505 07:53:59.853039 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 4783dd0b2fbe40219852d850fc0d9a82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4783dd0b2fbe40219852d850fc0d9a82 already in progress: creating tablet
W20260505 07:53:59.853294 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet f121fdd001ec4ea38ccd24a6afd9e51c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f121fdd001ec4ea38ccd24a6afd9e51c already in progress: creating tablet
W20260505 07:53:59.853530 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 472441bd29704be59aabb0f7a4964bb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 472441bd29704be59aabb0f7a4964bb0 already in progress: creating tablet
I20260505 07:53:59.854184 749 tablet_service.cc:1558] Processing DeleteTablet for tablet e85b44337f62494c997c0008fc492aab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.854094 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_9c422539fda7423c820e2450a3f7c8db"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.854447 1175 tablet_replica.cc:333] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:53:59.854606 751 tablet_service.cc:1511] Processing CreateTablet for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (DEFAULT_TABLE table=default.Table_7d1747ea770445058549830f424a7aa3 [id=438fe94f1aab49919a5aae9ccab8f491]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.854969 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f207a3ffbc94b01abd9e9d6837f5c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.854939 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ec95dd1d091844c3adb51935b2e48795"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.855867 755 tablet_service.cc:1511] Processing CreateTablet for tablet d5083f98e8e14ccf99a19cdf2bc59018 (DEFAULT_TABLE table=default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.856176 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5083f98e8e14ccf99a19cdf2bc59018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.856505 1181 tablet_bootstrap.cc:492] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:59.856626 1181 ts_tablet_manager.cc:1403] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.007s
I20260505 07:53:59.857556 1181 raft_consensus.cc:359] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.857753 1181 raft_consensus.cc:385] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.857846 1181 raft_consensus.cc:740] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.857870 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_438e1bb5f56946368851b8f2e8a7fd72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.858062 1181 consensus_queue.cc:260] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.859079 1181 ts_tablet_manager.cc:1434] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:53:59.859359 1181 tablet_bootstrap.cc:492] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:53:59.860018 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ec95dd1d091844c3adb51935b2e48795 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.860535 760 tablet_service.cc:1511] Processing CreateTablet for tablet 2045202714c14aa388d1e8bc18e2063d (DEFAULT_TABLE table=default.Table_657cc0ba6f494479bd433c8c0279c734 [id=f191fbb56e234f10a9f94b2b998781bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.860894 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2045202714c14aa388d1e8bc18e2063d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:59.862851 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c422539fda7423c820e2450a3f7c8db in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.862957 1181 tablet_bootstrap.cc:654] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:53:59.864006 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_438e1bb5f56946368851b8f2e8a7fd72 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.867872 758 tablet_service.cc:1511] Processing CreateTablet for tablet 4ef22101fe014e21adea6e10f4b550e7 (DEFAULT_TABLE table=default.Table_89965306827247e99ff9f34d13e63c54 [id=567b03439f3a412aa6c1a4f4bac48f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.868201 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4ef22101fe014e21adea6e10f4b550e7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:59.868885 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f842d8ac6b2142ffa69945c6087be34b consensus state has no leader
I20260505 07:53:59.870358 1181 tablet_bootstrap.cc:492] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:59.873147 1181 ts_tablet_manager.cc:1403] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.001s
I20260505 07:53:59.874902 1181 raft_consensus.cc:359] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.875458 1181 raft_consensus.cc:385] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.875965 1181 raft_consensus.cc:740] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.876432 1181 consensus_queue.cc:260] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.878629 1181 ts_tablet_manager.cc:1434] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:53:59.879313 1181 tablet_bootstrap.cc:492] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:59.881884 1181 tablet_bootstrap.cc:654] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.897042 1172 tablet_bootstrap.cc:492] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:53:59.897473 1172 ts_tablet_manager.cc:1403] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.061s user 0.012s sys 0.008s
I20260505 07:53:59.905148 781 raft_consensus.cc:2468] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:53:59.905818 988 leader_election.cc:304] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:53:59.906178 1544 raft_consensus.cc:2804] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:53:59.906217 596 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 571257bbb7e94315b60d84d6aa4917b1 (table default.Table_4ab0ac01b8bf49ef96683dca41dfadd4 [id=6cb8bc8043474bad9f4278b7eef7c71a]). Expected version 3 got 0
I20260505 07:53:59.909149 1175 raft_consensus.cc:2243] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:53:59.909682 1175 raft_consensus.cc:2272] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:53:59.910615 1175 ts_tablet_manager.cc:1916] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:53:59.914194 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 9044763bed1a41739ecce4def4cca1fe
I20260505 07:53:59.914439 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 2706bc9b9e02414693062deff5cfe81b (DEFAULT_TABLE table=default.Table_0410f2bbd0504738ba4aa1f7412e3405 [id=617f784f561c4c9c93be1bdd500fc4d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.916150 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 9044763bed1a41739ecce4def4cca1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.917567 1017 tablet_service.cc:1511] Processing CreateTablet for tablet abaa596871bb4a82aefc317932e91d35 (DEFAULT_TABLE table=default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d [id=1fcae33a3a2d48e7b2000cc7b906c7f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.917649 594 master_service.cc:954] ReplaceTablet: received request to replace tablet a7fe9c7199cc4039ae877dc3d4aab17e from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:59.919075 1181 tablet_bootstrap.cc:492] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:53:59.919239 1181 ts_tablet_manager.cc:1403] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.040s user 0.010s sys 0.011s
I20260505 07:53:59.919934 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_8f7c0f8991b1415abce7b31407477b52"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.920064 1181 raft_consensus.cc:359] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.920245 1181 raft_consensus.cc:385] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.920342 1181 raft_consensus.cc:740] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
W20260505 07:53:59.920809 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f7c0f8991b1415abce7b31407477b52 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.922689 1181 consensus_queue.cc:260] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.923035 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abaa596871bb4a82aefc317932e91d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.924814 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet a7fe9c7199cc4039ae877dc3d4aab17e
I20260505 07:53:59.925524 594 catalog_manager.cc:6741] ReplaceTablet: tablet a7fe9c7199cc4039ae877dc3d4aab17e deleted and replaced by tablet 0e08cf03ea0b495a94c6566ea6c8caad
I20260505 07:53:59.925748 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e08cf03ea0b495a94c6566ea6c8caad) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.927296 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 9044763bed1a41739ecce4def4cca1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.927337 749 tablet_service.cc:1511] Processing CreateTablet for tablet abaa596871bb4a82aefc317932e91d35 (DEFAULT_TABLE table=default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d [id=1fcae33a3a2d48e7b2000cc7b906c7f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.929661 746 tablet_service.cc:1511] Processing CreateTablet for tablet 2706bc9b9e02414693062deff5cfe81b (DEFAULT_TABLE table=default.Table_0410f2bbd0504738ba4aa1f7412e3405 [id=617f784f561c4c9c93be1bdd500fc4d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.931299 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2706bc9b9e02414693062deff5cfe81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.931684 1181 ts_tablet_manager.cc:1434] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260505 07:53:59.931802 757 tablet_service.cc:1558] Processing DeleteTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e08cf03ea0b495a94c6566ea6c8caad) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.932274 756 tablet_service.cc:1511] Processing CreateTablet for tablet b4515a0ff8304c69b9aa030e9b14d43f (DEFAULT_TABLE table=default.Table_3621941a687641ecbcaa200aaefbe112 [id=2164fb49c5204df0ac8c95667d342af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.932780 1181 tablet_bootstrap.cc:492] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:53:59.934438 1181 tablet_bootstrap.cc:654] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.934890 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0b3a73b640a644579fa864cce1a7ecbe" } modify_external_catalogs: true
I20260505 07:53:59.935169 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_0b3a73b640a644579fa864cce1a7ecbe" } modify_external_catalogs: true
I20260505 07:53:59.938289 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2872bda04bf547648372a8f4bf5dc3ce" } modify_external_catalogs: true
I20260505 07:53:59.938464 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2872bda04bf547648372a8f4bf5dc3ce" } modify_external_catalogs: true
I20260505 07:53:59.939853 599 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_73ba90aede024514abb230661a44a4ae" } modify_external_catalogs: true
I20260505 07:53:59.940038 599 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_73ba90aede024514abb230661a44a4ae" } modify_external_catalogs: true
I20260505 07:53:59.940075 598 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 571257bbb7e94315b60d84d6aa4917b1 (table default.Table_4ab0ac01b8bf49ef96683dca41dfadd4 [id=6cb8bc8043474bad9f4278b7eef7c71a]). Expected version 3 got 0
I20260505 07:53:59.940274 598 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
I20260505 07:53:59.941324 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_44ead87f92584e6e98e2c5f69056591a" } modify_external_catalogs: true
I20260505 07:53:59.941514 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_44ead87f92584e6e98e2c5f69056591a" } modify_external_catalogs: true
I20260505 07:53:59.942986 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet faf1140fe05e4f9493563ab8464c582b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.943223 1028 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ac06bb9d273457ead843f5dd754539c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.944104 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 36b33d265ba04e93993c7e0a4efea1a4
I20260505 07:53:59.945478 1028 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.947130 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ad8e6e942c73439fb64536b3ab1edc9c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:53:59.949867 599 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 0db162dfad2448e8ac428b3d3fbe17f7
I20260505 07:53:59.949899 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet f1058fcb9fa6457a85489049ca1d9ada
I20260505 07:53:59.951048 1028 tablet_service.cc:1558] Processing DeleteTablet for tablet f1058fcb9fa6457a85489049ca1d9ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.951270 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.951980 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 297537a92684425d8508f7b3b4c9d379
I20260505 07:53:59.952657 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 297537a92684425d8508f7b3b4c9d379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.953846 757 tablet_service.cc:1558] Processing DeleteTablet for tablet f1058fcb9fa6457a85489049ca1d9ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.954610 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 297537a92684425d8508f7b3b4c9d379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:53:59.955963 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad8e6e942c73439fb64536b3ab1edc9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:53:59.960772 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 6429919aba17447dbdb393176360bbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:53:59.961172 755 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.961903 1176 tablet_bootstrap.cc:492] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:53:59.962090 1176 ts_tablet_manager.cc:1403] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.122s user 0.002s sys 0.002s
I20260505 07:53:59.963651 1176 raft_consensus.cc:359] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.963886 1176 raft_consensus.cc:385] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.963975 1176 raft_consensus.cc:740] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.964253 1176 consensus_queue.cc:260] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:53:59.965580 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (DEFAULT_TABLE table=default.Table_3c9fe3aa59174b11a7f5f4488741deec [id=34b0c1456db3424da7730ffd60760ebb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.966080 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cbfb2345a1a4b218c2dddbae6667e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.975685 1017 tablet_service.cc:1511] Processing CreateTablet for tablet b4515a0ff8304c69b9aa030e9b14d43f (DEFAULT_TABLE table=default.Table_3621941a687641ecbcaa200aaefbe112 [id=2164fb49c5204df0ac8c95667d342af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.976235 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4515a0ff8304c69b9aa030e9b14d43f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:53:59.976581 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 3272555a8c814c0ebe61ca32d715212b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.977460 1366 raft_consensus.cc:493] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:53:59.977661 1366 raft_consensus.cc:515] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.977845 1172 raft_consensus.cc:359] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.980228 890 tablet_service.cc:1511] Processing CreateTablet for tablet 4b528d6c36ac4920b6ea0a6e26cca269 (DEFAULT_TABLE table=default.Table_79ed240074b343fabbdd206872328277 [id=7cbbeae00a3d4f85bcaef70b7aaa69de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.981555 1172 raft_consensus.cc:385] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:53:59.981582 597 master_service.cc:954] ReplaceTablet: received request to replace tablet bc9ae6404dd64bd8bf4898768e94e4f1 from {username='slave'} at 127.0.0.1:47238
I20260505 07:53:59.981679 1172 raft_consensus.cc:740] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:53:59.982426 1172 consensus_queue.cc:260] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:53:59.984264 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d113a872f9400c8f28d0239ca42377" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:53:59.984764 1366 leader_election.cc:290] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:53:59.985316 854 leader_election.cc:343] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:59.986373 893 tablet_service.cc:1558] Processing DeleteTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e08cf03ea0b495a94c6566ea6c8caad) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.986815 882 tablet_service.cc:1511] Processing CreateTablet for tablet abaa596871bb4a82aefc317932e91d35 (DEFAULT_TABLE table=default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d [id=1fcae33a3a2d48e7b2000cc7b906c7f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.987629 886 tablet_service.cc:1558] Processing DeleteTablet for tablet f92c53c68a4b4e319e98b84128e8e6a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.988821 883 tablet_service.cc:1511] Processing CreateTablet for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (DEFAULT_TABLE table=default.Table_3c9fe3aa59174b11a7f5f4488741deec [id=34b0c1456db3424da7730ffd60760ebb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.989080 1176 ts_tablet_manager.cc:1434] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.027s user 0.003s sys 0.000s
I20260505 07:53:59.989459 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d113a872f9400c8f28d0239ca42377" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:53:59.989718 1176 tablet_bootstrap.cc:492] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:53:59.991734 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:53:59.992118 1176 tablet_bootstrap.cc:654] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:53:59.993331 758 tablet_service.cc:1511] Processing CreateTablet for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (DEFAULT_TABLE table=default.Table_3c9fe3aa59174b11a7f5f4488741deec [id=34b0c1456db3424da7730ffd60760ebb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.994208 895 tablet_service.cc:1511] Processing CreateTablet for tablet 2706bc9b9e02414693062deff5cfe81b (DEFAULT_TABLE table=default.Table_0410f2bbd0504738ba4aa1f7412e3405 [id=617f784f561c4c9c93be1bdd500fc4d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.995358 884 tablet_service.cc:1558] Processing DeleteTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.996317 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 9044763bed1a41739ecce4def4cca1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.995381 893 tablet_service.cc:1511] Processing CreateTablet for tablet b4515a0ff8304c69b9aa030e9b14d43f (DEFAULT_TABLE table=default.Table_3621941a687641ecbcaa200aaefbe112 [id=2164fb49c5204df0ac8c95667d342af5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.997067 886 tablet_service.cc:1558] Processing DeleteTablet for tablet f1058fcb9fa6457a85489049ca1d9ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.997617 897 tablet_service.cc:1558] Processing DeleteTablet for tablet 297537a92684425d8508f7b3b4c9d379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.997594 1027 tablet_service.cc:1511] Processing CreateTablet for tablet e8bf49b787ef4de095448751464708a4 (DEFAULT_TABLE table=default.Table_b00ce8f351ce43edb2431c6e621d329e [id=39b432d60dc64a5aa01808a557238a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:53:59.998149 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bf49b787ef4de095448751464708a4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:53:59.998562 856 leader_election.cc:343] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:53:59.998716 856 leader_election.cc:304] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:53:59.999385 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:53:59.999567 897 tablet_service.cc:1511] Processing CreateTablet for tablet e8bf49b787ef4de095448751464708a4 (DEFAULT_TABLE table=default.Table_b00ce8f351ce43edb2431c6e621d329e [id=39b432d60dc64a5aa01808a557238a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.000481 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cbfb2345a1a4b218c2dddbae6667e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.001305 1028 tablet_service.cc:1511] Processing CreateTablet for tablet a890d5e3f1af49b5aaa28c6c51cb3874 (DEFAULT_TABLE table=default.Table_eb1e569ff0a74997b956acbad82f6c49 [id=e996e55aed7c4c08bb2fd8a6fc358155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.001533 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4515a0ff8304c69b9aa030e9b14d43f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.001960 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abaa596871bb4a82aefc317932e91d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.002031 886 tablet_service.cc:1511] Processing CreateTablet for tablet a890d5e3f1af49b5aaa28c6c51cb3874 (DEFAULT_TABLE table=default.Table_eb1e569ff0a74997b956acbad82f6c49 [id=e996e55aed7c4c08bb2fd8a6fc358155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.002413 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bf49b787ef4de095448751464708a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.003808 887 tablet_service.cc:1511] Processing CreateTablet for tablet 29a82ed2b87c487aac84194f2dcb0d6b (DEFAULT_TABLE table=default.Table_7b4336be586a4fc1a8dada3651641fd9 [id=057b807a46cb43b4a60dc61212a8b8a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.004995 1366 raft_consensus.cc:2749] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.005412 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2706bc9b9e02414693062deff5cfe81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.009143 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29a82ed2b87c487aac84194f2dcb0d6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.009907 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2706bc9b9e02414693062deff5cfe81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.010143 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abaa596871bb4a82aefc317932e91d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.010457 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4515a0ff8304c69b9aa030e9b14d43f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.012104 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_5fd319488e1d4b9da8f13212d678a934" } new_table_name: "default.Table_db5c1accbc4540428bd12b85b71b9e76" modify_external_catalogs: true
I20260505 07:54:00.015309 754 tablet_service.cc:1511] Processing CreateTablet for tablet 29a82ed2b87c487aac84194f2dcb0d6b (DEFAULT_TABLE table=default.Table_7b4336be586a4fc1a8dada3651641fd9 [id=057b807a46cb43b4a60dc61212a8b8a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.015838 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29a82ed2b87c487aac84194f2dcb0d6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.020033 751 tablet_service.cc:1511] Processing CreateTablet for tablet a890d5e3f1af49b5aaa28c6c51cb3874 (DEFAULT_TABLE table=default.Table_eb1e569ff0a74997b956acbad82f6c49 [id=e996e55aed7c4c08bb2fd8a6fc358155]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.020893 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a890d5e3f1af49b5aaa28c6c51cb3874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.024299 761 tablet_service.cc:1511] Processing CreateTablet for tablet e8bf49b787ef4de095448751464708a4 (DEFAULT_TABLE table=default.Table_b00ce8f351ce43edb2431c6e621d329e [id=39b432d60dc64a5aa01808a557238a4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.024889 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bf49b787ef4de095448751464708a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.038234 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 7585f00119e14006990ed551667b4486 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.038473 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a890d5e3f1af49b5aaa28c6c51cb3874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.042119 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a890d5e3f1af49b5aaa28c6c51cb3874. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.042182 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 83e62b7af9b847b7a9005e4345905c0d consensus state has no leader
I20260505 07:54:00.042609 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cbfb2345a1a4b218c2dddbae6667e47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.045121 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_d023acedd7c842ebb396d7354c41b490" } new_table_name: "default.Table_c3436374329a44868874945e540a1d9b" modify_external_catalogs: true
I20260505 07:54:00.047880 901 tablet_service.cc:1511] Processing CreateTablet for tablet e82e9ae417784b7e83f4843b129d0580 (DEFAULT_TABLE table=default.Table_9f8b49c83f7843318fd7a4645786e1d4 [id=a12c6d5dd696413490da09bf773c9a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.049329 884 tablet_service.cc:1511] Processing CreateTablet for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (DEFAULT_TABLE table=default.Table_5f0d98b4110b4596a6de3749d10302c9 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.049875 898 tablet_service.cc:1511] Processing CreateTablet for tablet 60772a9697ba417b879dc6025eeb8eec (DEFAULT_TABLE table=default.Table_ee5371f299e44106859f3663fe6052b7 [id=a4d53a8c6539419daba67a153292bb0c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:00.052625 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f84f464e83cb412eade9978ea99f6685 consensus state has no leader
I20260505 07:54:00.053426 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f93a81c5dfe740819f0f6ea5bf7f1067"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.054504 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f93a81c5dfe740819f0f6ea5bf7f1067 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.054476 756 tablet_service.cc:1511] Processing CreateTablet for tablet e82e9ae417784b7e83f4843b129d0580 (DEFAULT_TABLE table=default.Table_9f8b49c83f7843318fd7a4645786e1d4 [id=a12c6d5dd696413490da09bf773c9a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.055033 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f40f1996d8454947afd1ff280e2b5fc1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.056277 599 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_cf18ef8e6a56482883dba2744997604a" } new_table_name: "default.Table_95fae21889c84b61ad67610d209031b2" modify_external_catalogs: true
W20260505 07:54:00.055985 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f40f1996d8454947afd1ff280e2b5fc1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:00.064851 599 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 76939b7978ee4f7ea0ad29312de2a31a consensus state has no leader
I20260505 07:54:00.066238 896 tablet_service.cc:1511] Processing CreateTablet for tablet e7887c3462f74740b314e3bec36e280d (DEFAULT_TABLE table=default.Table_8ef2c24cce2148ecb1409ecaa29a8130 [id=c82e4e1fd6004306906c95957b86620f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.068475 885 tablet_service.cc:1511] Processing CreateTablet for tablet 70051011024f4971bd0166d335d32393 (DEFAULT_TABLE table=default.Table_6b9d99bdf8624bba90ba61af22f36dc2 [id=6fa686898f454d2aa691814ad1284d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.069500 754 tablet_service.cc:1511] Processing CreateTablet for tablet 60772a9697ba417b879dc6025eeb8eec (DEFAULT_TABLE table=default.Table_ee5371f299e44106859f3663fe6052b7 [id=a4d53a8c6539419daba67a153292bb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.070137 751 tablet_service.cc:1511] Processing CreateTablet for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (DEFAULT_TABLE table=default.Table_5f0d98b4110b4596a6de3749d10302c9 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.072466 899 tablet_service.cc:1511] Processing CreateTablet for tablet 0bd2e2514c72419390c1bb9bd13f1fb3 (DEFAULT_TABLE table=default.Table_f75ef4b5e6ae433f8c6bee72fc30eaf2 [id=f35c58e2d1f54d3ba63a303a0648cebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.073077 891 tablet_service.cc:1511] Processing CreateTablet for tablet 0e9a98243da14d43bc649bb19aed4f0c (DEFAULT_TABLE table=default.Table_21673305a26a4342a0be11d98e875447 [id=76c4943b63be49fdaf6ee51abe09b9b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.079957 889 tablet_service.cc:1511] Processing CreateTablet for tablet 72da22d790624b07bfca6f6b6301c925 (DEFAULT_TABLE table=default.Table_6c7c80f7fea34238b6cf32a173f304b5 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.080652 894 tablet_service.cc:1511] Processing CreateTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e (DEFAULT_TABLE table=default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.082479 1175 ts_tablet_manager.cc:1929] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.083019 1181 tablet_bootstrap.cc:492] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:00.083050 1171 ts_tablet_manager.cc:1929] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.083189 1171 log.cc:1199] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/1852f7f4ff594639908b03730b68fe86
I20260505 07:54:00.083642 1171 ts_tablet_manager.cc:1950] T 1852f7f4ff594639908b03730b68fe86 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:00.084189 1178 ts_tablet_manager.cc:1929] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.084331 1178 log.cc:1199] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/e1a9420a81844d2092381123d9297661
I20260505 07:54:00.084697 1178 ts_tablet_manager.cc:1950] T e1a9420a81844d2092381123d9297661 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:00.084159 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 29a82ed2b87c487aac84194f2dcb0d6b (DEFAULT_TABLE table=default.Table_7b4336be586a4fc1a8dada3651641fd9 [id=057b807a46cb43b4a60dc61212a8b8a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.084889 1181 ts_tablet_manager.cc:1403] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.150s user 0.027s sys 0.019s
I20260505 07:54:00.085114 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29a82ed2b87c487aac84194f2dcb0d6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.091929 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (DEFAULT_TABLE table=default.Table_5f0d98b4110b4596a6de3749d10302c9 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.092087 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e8f9f7b30f4912af916c15f9ae25d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.092648 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e9a98243da14d43bc649bb19aed4f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.093171 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bd2e2514c72419390c1bb9bd13f1fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.093736 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6546745e894545ff9c5f91bcc6e1a41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.094015 1171 tablet_replica.cc:333] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:00.094149 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72da22d790624b07bfca6f6b6301c925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.094271 1171 raft_consensus.cc:2243] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.094491 1171 raft_consensus.cc:2272] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:00.094604 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b528d6c36ac4920b6ea0a6e26cca269. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.095206 1171 ts_tablet_manager.cc:1916] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.102681 1172 ts_tablet_manager.cc:1434] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.205s user 0.020s sys 0.024s
I20260505 07:54:00.103070 1172 tablet_bootstrap.cc:492] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:00.104810 1172 tablet_bootstrap.cc:654] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.105676 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70051011024f4971bd0166d335d32393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.106532 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60772a9697ba417b879dc6025eeb8eec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.107002 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7887c3462f74740b314e3bec36e280d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.107445 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e82e9ae417784b7e83f4843b129d0580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.110949 1569 raft_consensus.cc:493] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.111176 1569 raft_consensus.cc:515] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.112468 1172 tablet_bootstrap.cc:492] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:00.112629 1172 ts_tablet_manager.cc:1403] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20260505 07:54:00.113080 1569 leader_election.cc:290] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.113379 1172 raft_consensus.cc:359] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.113579 1172 raft_consensus.cc:385] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.113680 1172 raft_consensus.cc:740] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.114022 1175 log.cc:1199] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/e85b44337f62494c997c0008fc492aab
I20260505 07:54:00.114365 1175 ts_tablet_manager.cc:1950] T e85b44337f62494c997c0008fc492aab P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:00.115298 1172 consensus_queue.cc:260] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.115852 1172 ts_tablet_manager.cc:1434] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:00.116120 1172 tablet_bootstrap.cc:492] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:00.091755 746 tablet_service.cc:1511] Processing CreateTablet for tablet e7887c3462f74740b314e3bec36e280d (DEFAULT_TABLE table=default.Table_8ef2c24cce2148ecb1409ecaa29a8130 [id=c82e4e1fd6004306906c95957b86620f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.104007 1176 tablet_bootstrap.cc:492] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:00.113535 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585f00119e14006990ed551667b4486" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:00.117307 854 leader_election.cc:343] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.117565 1176 ts_tablet_manager.cc:1403] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.128s user 0.004s sys 0.000s
I20260505 07:54:00.117802 1172 tablet_bootstrap.cc:654] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.117972 749 tablet_service.cc:1511] Processing CreateTablet for tablet 70051011024f4971bd0166d335d32393 (DEFAULT_TABLE table=default.Table_6b9d99bdf8624bba90ba61af22f36dc2 [id=6fa686898f454d2aa691814ad1284d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.122097 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7887c3462f74740b314e3bec36e280d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.122637 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60772a9697ba417b879dc6025eeb8eec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.123068 1175 tablet_replica.cc:333] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:00.123160 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e8f9f7b30f4912af916c15f9ae25d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.123256 1175 raft_consensus.cc:2243] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.123296 1172 tablet_bootstrap.cc:492] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:00.123443 1172 ts_tablet_manager.cc:1403] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260505 07:54:00.124012 1175 raft_consensus.cc:2272] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:00.124645 1172 raft_consensus.cc:359] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.124852 1172 raft_consensus.cc:385] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.124939 1172 raft_consensus.cc:740] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.125178 1172 consensus_queue.cc:260] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.125662 1172 ts_tablet_manager.cc:1434] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:00.126817 1175 ts_tablet_manager.cc:1916] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.124008 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e82e9ae417784b7e83f4843b129d0580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.087728 1181 raft_consensus.cc:359] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.129014 761 tablet_service.cc:1511] Processing CreateTablet for tablet 0bd2e2514c72419390c1bb9bd13f1fb3 (DEFAULT_TABLE table=default.Table_f75ef4b5e6ae433f8c6bee72fc30eaf2 [id=f35c58e2d1f54d3ba63a303a0648cebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.129848 1172 tablet_bootstrap.cc:492] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:00.131489 1172 tablet_bootstrap.cc:654] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.132328 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bd2e2514c72419390c1bb9bd13f1fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.141230 1172 tablet_bootstrap.cc:492] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:00.141381 1172 ts_tablet_manager.cc:1403] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260505 07:54:00.142100 1172 raft_consensus.cc:359] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:00.142280 1172 raft_consensus.cc:385] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.142371 1172 raft_consensus.cc:740] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.145570 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585f00119e14006990ed551667b4486" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:00.146569 856 leader_election.cc:343] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.146689 856 leader_election.cc:304] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.142594 1172 consensus_queue.cc:260] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:00.148813 1366 raft_consensus.cc:2749] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.149397 1172 ts_tablet_manager.cc:1434] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260505 07:54:00.167734 1181 raft_consensus.cc:385] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.167860 1181 raft_consensus.cc:740] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.167960 1031 tablet_service.cc:1511] Processing CreateTablet for tablet e7887c3462f74740b314e3bec36e280d (DEFAULT_TABLE table=default.Table_8ef2c24cce2148ecb1409ecaa29a8130 [id=c82e4e1fd6004306906c95957b86620f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.168129 1025 tablet_service.cc:1511] Processing CreateTablet for tablet e82e9ae417784b7e83f4843b129d0580 (DEFAULT_TABLE table=default.Table_9f8b49c83f7843318fd7a4645786e1d4 [id=a12c6d5dd696413490da09bf773c9a2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.168251 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e8f9f7b30f4912af916c15f9ae25d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.169378 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 258) took 2488 ms. Trace:
I20260505 07:54:00.169056 1181 consensus_queue.cc:260] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.170217 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7887c3462f74740b314e3bec36e280d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.170576 1181 ts_tablet_manager.cc:1434] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.086s user 0.001s sys 0.001s
I20260505 07:54:00.171289 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 1852f7f4ff594639908b03730b68fe86 (table default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]) successfully deleted
I20260505 07:54:00.172369 1181 tablet_bootstrap.cc:492] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:00.173599 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 70051011024f4971bd0166d335d32393 (DEFAULT_TABLE table=default.Table_6b9d99bdf8624bba90ba61af22f36dc2 [id=6fa686898f454d2aa691814ad1284d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.177546 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 0bd2e2514c72419390c1bb9bd13f1fb3 (DEFAULT_TABLE table=default.Table_f75ef4b5e6ae433f8c6bee72fc30eaf2 [id=f35c58e2d1f54d3ba63a303a0648cebf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.177835 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e82e9ae417784b7e83f4843b129d0580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.183478 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet e85b44337f62494c997c0008fc492aab (table default.Table_b8745d36d43a4f42a5b3d43a37c21e55 [id=06480a024264477596ab9058373c273b]) successfully deleted
I20260505 07:54:00.185446 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 72da22d790624b07bfca6f6b6301c925 (DEFAULT_TABLE table=default.Table_6c7c80f7fea34238b6cf32a173f304b5 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.186172 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70051011024f4971bd0166d335d32393. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.187682 588 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 75292.7us
W20260505 07:54:00.187803 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 9044763bed1a41739ecce4def4cca1fe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9044763bed1a41739ecce4def4cca1fe already in progress: creating tablet
W20260505 07:54:00.187928 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 02412333633a4cdebe95a144fa6c363c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02412333633a4cdebe95a144fa6c363c already in progress: creating tablet
W20260505 07:54:00.188059 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet a7fe9c7199cc4039ae877dc3d4aab17e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7fe9c7199cc4039ae877dc3d4aab17e already in progress: creating tablet
W20260505 07:54:00.188185 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 8b406c4c6ba741bc85eef54c6f576459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b406c4c6ba741bc85eef54c6f576459 already in progress: creating tablet
W20260505 07:54:00.188274 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet f1058fcb9fa6457a85489049ca1d9ada because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1058fcb9fa6457a85489049ca1d9ada already in progress: creating tablet
I20260505 07:54:00.188324 1022 tablet_service.cc:1511] Processing CreateTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e (DEFAULT_TABLE table=default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:00.188390 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a7d8265c05b74b28959d0de41fad1fde because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7d8265c05b74b28959d0de41fad1fde already in progress: creating tablet
W20260505 07:54:00.188500 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 297537a92684425d8508f7b3b4c9d379 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 297537a92684425d8508f7b3b4c9d379 already in progress: creating tablet
W20260505 07:54:00.188608 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 3075e87e078f4d2aa1d329d18902354b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3075e87e078f4d2aa1d329d18902354b already in progress: creating tablet
I20260505 07:54:00.188594 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 0e9a98243da14d43bc649bb19aed4f0c (DEFAULT_TABLE table=default.Table_21673305a26a4342a0be11d98e875447 [id=76c4943b63be49fdaf6ee51abe09b9b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.188920 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72da22d790624b07bfca6f6b6301c925. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.188885 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 36b33d265ba04e93993c7e0a4efea1a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36b33d265ba04e93993c7e0a4efea1a4 already in progress: creating tablet
I20260505 07:54:00.189857 596 master_service.cc:954] ReplaceTablet: received request to replace tablet 6ca2852cee84476dbe5e5072a43087a1 from {username='slave'} at 127.0.0.1:47238
I20260505 07:54:00.190023 597 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_5613a0267c774633aabe68f172339eea" } new_table_name: "default.Table_1899c1d796874753b600a2185d9e3088" modify_external_catalogs: true
I20260505 07:54:00.190490 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_2847b3b378404489a41b1c874e251a83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.191010 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2847b3b378404489a41b1c874e251a83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.191256 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_cc718440246d4ddfb49faf4602910188"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.191921 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cc718440246d4ddfb49faf4602910188 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.192021 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6c7c80f7fea34238b6cf32a173f304b5" } new_table_name: "default.Table_201f8409684e49be8bc5cd5249606250" modify_external_catalogs: true
I20260505 07:54:00.191987 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_dedf48e6bbc941f6b329f367499612f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.192646 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dedf48e6bbc941f6b329f367499612f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.193169 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bd2e2514c72419390c1bb9bd13f1fb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.193485 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_4ce294c5bd3f49699d1e194c53ff9c07"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.193645 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1aa571f4a9f54c8db6250dadf285b36b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.194278 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ce294c5bd3f49699d1e194c53ff9c07 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:00.194489 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1aa571f4a9f54c8db6250dadf285b36b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.195485 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_a31277b1c1ca4863976cb9726e83f13a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.196007 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a31277b1c1ca4863976cb9726e83f13a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.196733 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_9f8b49c83f7843318fd7a4645786e1d4" } modify_external_catalogs: true
I20260505 07:54:00.196979 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_9f8b49c83f7843318fd7a4645786e1d4" } modify_external_catalogs: true
I20260505 07:54:00.198153 755 tablet_service.cc:1511] Processing CreateTablet for tablet 72da22d790624b07bfca6f6b6301c925 (DEFAULT_TABLE table=default.Table_6c7c80f7fea34238b6cf32a173f304b5 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.198555 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72da22d790624b07bfca6f6b6301c925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.204706 761 tablet_service.cc:1511] Processing CreateTablet for tablet 0e9a98243da14d43bc649bb19aed4f0c (DEFAULT_TABLE table=default.Table_21673305a26a4342a0be11d98e875447 [id=76c4943b63be49fdaf6ee51abe09b9b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.205029 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e9a98243da14d43bc649bb19aed4f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.205812 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6546745e894545ff9c5f91bcc6e1a41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.207226 1181 tablet_bootstrap.cc:654] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.208002 1172 tablet_bootstrap.cc:492] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:00.209859 1172 tablet_bootstrap.cc:654] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.211666 1178 rpcz_store.cc:276] 0505 07:53:57.666685 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.666729 (+ 44us) service_pool.cc:224] Handling call
0505 07:53:59.835064 (+2168335us) ts_tablet_manager.cc:1151] Deleting tablet e1a9420a81844d2092381123d9297661
0505 07:53:59.835072 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:53:59.935415 (+100343us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.083994 (+148579us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.155071 (+ 71077us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2168174}
I20260505 07:54:00.212203 1178 tablet_replica.cc:333] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:00.212487 1178 raft_consensus.cc:2243] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.212673 1178 raft_consensus.cc:2272] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:00.213342 1178 ts_tablet_manager.cc:1916] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.213516 1567 raft_consensus.cc:493] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.213676 1567 raft_consensus.cc:515] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:00.214440 1567 leader_election.cc:290] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:00.215533 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 60772a9697ba417b879dc6025eeb8eec (DEFAULT_TABLE table=default.Table_ee5371f299e44106859f3663fe6052b7 [id=a4d53a8c6539419daba67a153292bb0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.216171 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet e1a9420a81844d2092381123d9297661 (table default.Table_422080d5c75f422fb88c26b76ffc61c4 [id=0d8de70d77c54c2491b257d09c8474c8]) successfully deleted
I20260505 07:54:00.218307 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6429919aba17447dbdb393176360bbb0" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:00.218722 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 72da22d790624b07bfca6f6b6301c925 consensus state has no leader
I20260505 07:54:00.219516 593 master_service.cc:954] ReplaceTablet: received request to replace tablet ff7ed926a3ff4e7993864de734ca3c50 from {username='slave'} at 127.0.0.1:47238
W20260505 07:54:00.219695 597 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5683d7366de84653ace56f0edcd3818c consensus state has no leader
I20260505 07:54:00.219223 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70051011024f4971bd0166d335d32393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.223675 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e9a98243da14d43bc649bb19aed4f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.223201 755 tablet_service.cc:1511] Processing CreateTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e (DEFAULT_TABLE table=default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.225520 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6546745e894545ff9c5f91bcc6e1a41e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.226892 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 976359601ad74b149696dc7f555b550c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:54:00.228495 988 leader_election.cc:343] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.229993 1176 raft_consensus.cc:359] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.230322 1176 raft_consensus.cc:385] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.230580 1176 raft_consensus.cc:740] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.230969 1176 consensus_queue.cc:260] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.234218 1176 ts_tablet_manager.cc:1434] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.117s user 0.002s sys 0.000s
I20260505 07:54:00.238103 1569 raft_consensus.cc:493] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.238256 1569 raft_consensus.cc:515] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:00.238871 1569 leader_election.cc:290] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:00.240082 600 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_e2720de78a7742d5829d7db2ef803f0d" } modify_external_catalogs: true
I20260505 07:54:00.240307 600 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_e2720de78a7742d5829d7db2ef803f0d" } modify_external_catalogs: true
I20260505 07:54:00.241221 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ca13a11f3fd249eabadc7213e7318acd" } modify_external_catalogs: true
I20260505 07:54:00.241427 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ca13a11f3fd249eabadc7213e7318acd" } modify_external_catalogs: true
I20260505 07:54:00.242089 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_876e4bf0b9054b8e836299103ac03ef7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.242772 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_876e4bf0b9054b8e836299103ac03ef7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.246266 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6bd554849539457bb38dbbef32deb609" } modify_external_catalogs: true
I20260505 07:54:00.246492 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_6bd554849539457bb38dbbef32deb609" } modify_external_catalogs: true
I20260505 07:54:00.267858 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60772a9697ba417b879dc6025eeb8eec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.268914 991 leader_election.cc:343] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 6429919aba17447dbdb393176360bbb0
I20260505 07:54:00.269097 991 leader_election.cc:304] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.271399 1567 raft_consensus.cc:493] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.239238 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd36b5a84b4442fb76e1610d8af4c07" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:00.247354 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.247746 1175 ts_tablet_manager.cc:1929] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.273056 1175 log.cc:1199] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/3272555a8c814c0ebe61ca32d715212b
I20260505 07:54:00.271139 1544 raft_consensus.cc:697] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:00.273455 1175 ts_tablet_manager.cc:1950] T 3272555a8c814c0ebe61ca32d715212b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:00.275298 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 3272555a8c814c0ebe61ca32d715212b (table default.Table_70e3149cee264c889c7677c3757ef266 [id=27e306dc7bda4ca0b28f72e8636b91d5]) successfully deleted
W20260505 07:54:00.276037 854 leader_election.cc:343] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.277001 1567 raft_consensus.cc:515] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:54:00.277256 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 0db162dfad2448e8ac428b3d3fbe17f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0db162dfad2448e8ac428b3d3fbe17f7 already in progress: creating tablet
W20260505 07:54:00.277551 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b0cf443e3044e56a26b0a8a61c9cf7d already in progress: creating tablet
I20260505 07:54:00.280093 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd36b5a84b4442fb76e1610d8af4c07" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:00.280568 1567 leader_election.cc:290] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.280853 1544 consensus_queue.cc:237] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.283049 1567 raft_consensus.cc:493] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.283179 1567 raft_consensus.cc:515] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.284274 1171 ts_tablet_manager.cc:1929] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.284421 1171 log.cc:1199] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/ccdde3e5a13e436d9f2328831e155adb
I20260505 07:54:00.284793 1171 ts_tablet_manager.cc:1950] T ccdde3e5a13e436d9f2328831e155adb P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
W20260505 07:54:00.285236 988 leader_election.cc:343] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 25a74b65d5d6459293ab4461f4d34cf2
W20260505 07:54:00.285590 856 leader_election.cc:343] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.285722 856 leader_election.cc:304] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.286300 1569 raft_consensus.cc:2749] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.286494 1171 tablet_replica.cc:333] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:00.286509 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet ccdde3e5a13e436d9f2328831e155adb (table default.Table_efeac090fb094bdf82f547af366d562c [id=047ae44eaf5e48ba8b11e90f6d5f8119]) successfully deleted
W20260505 07:54:00.286734 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 4f3e076892234c1494b1a5c13cbfd2e2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f3e076892234c1494b1a5c13cbfd2e2 already in progress: creating tablet
I20260505 07:54:00.286783 1171 raft_consensus.cc:2243] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:00.286933 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 0459f2d45a2d44f1a73e0a0fff53866c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0459f2d45a2d44f1a73e0a0fff53866c already in progress: creating tablet
I20260505 07:54:00.286962 1171 raft_consensus.cc:2272] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:00.287148 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 4783dd0b2fbe40219852d850fc0d9a82 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4783dd0b2fbe40219852d850fc0d9a82 already in progress: creating tablet
I20260505 07:54:00.287487 1566 raft_consensus.cc:2749] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.287933 1176 tablet_bootstrap.cc:492] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:00.288622 1567 leader_election.cc:290] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.288661 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571257bbb7e94315b60d84d6aa4917b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:00.290763 1176 tablet_bootstrap.cc:654] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.292500 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c22dca26148c48f882eeea850f50bea4" } modify_external_catalogs: true
I20260505 07:54:00.292693 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_8c264d1fb97d491bb94ba720eef63e01"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.293236 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c264d1fb97d491bb94ba720eef63e01 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.293562 598 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2f8e689cd77249e9a31aa6d90f55b4ae" } new_table_name: "default.Table_1ff5286907c6464c8addf47c70eef7ef" modify_external_catalogs: true
I20260505 07:54:00.292693 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c22dca26148c48f882eeea850f50bea4" } modify_external_catalogs: true
I20260505 07:54:00.295838 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet e82e9ae417784b7e83f4843b129d0580
I20260505 07:54:00.296543 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 003cd4472b424aacbba8c5f89900e228 from {username='slave'} at 127.0.0.1:47238
I20260505 07:54:00.297037 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_438e1bb5f56946368851b8f2e8a7fd72" } new_table_name: "default.Table_55c9c867105547fc9295e1dc46d9a865" modify_external_catalogs: true
I20260505 07:54:00.298601 889 tablet_service.cc:1558] Processing DeleteTablet for tablet e82e9ae417784b7e83f4843b129d0580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.299544 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet e82e9ae417784b7e83f4843b129d0580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.301748 601 master_service.cc:954] ReplaceTablet: received request to replace tablet e010f23c4df647ab863e03c3cfed7898 from {username='slave'} at 127.0.0.1:47238
W20260505 07:54:00.302630 988 leader_election.cc:343] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.303663 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.303762 760 tablet_service.cc:1558] Processing DeleteTablet for tablet e82e9ae417784b7e83f4843b129d0580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.304006 1175 tablet_replica.cc:333] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:00.304219 1175 raft_consensus.cc:2243] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.304431 1175 raft_consensus.cc:2272] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:00.305289 1175 ts_tablet_manager.cc:1916] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.311877 1175 ts_tablet_manager.cc:1929] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.311990 1175 log.cc:1199] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/8b0cf443e3044e56a26b0a8a61c9cf7d
I20260505 07:54:00.312314 1175 ts_tablet_manager.cc:1950] T 8b0cf443e3044e56a26b0a8a61c9cf7d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:00.313760 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 8b0cf443e3044e56a26b0a8a61c9cf7d (table default.Table_f4c4ec691a3445b08b83d566bd437fb8 [id=740016f529e141b58f3ea08e9b984052]) successfully deleted
W20260505 07:54:00.314007 585 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e82e9ae417784b7e83f4843b129d0580 because the tablet was not found. No further retry: Not found: Tablet not found: e82e9ae417784b7e83f4843b129d0580
I20260505 07:54:00.317097 1181 tablet_bootstrap.cc:492] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:00.317269 1181 ts_tablet_manager.cc:1403] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.146s user 0.019s sys 0.024s
I20260505 07:54:00.318830 1172 tablet_bootstrap.cc:492] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:00.319053 1172 ts_tablet_manager.cc:1403] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.111s user 0.002s sys 0.014s
I20260505 07:54:00.320434 1172 raft_consensus.cc:359] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.320709 1172 raft_consensus.cc:385] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.320833 1172 raft_consensus.cc:740] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.320793 1181 raft_consensus.cc:359] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.320966 1181 raft_consensus.cc:385] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.321192 1172 consensus_queue.cc:260] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.322860 1172 ts_tablet_manager.cc:1434] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260505 07:54:00.323194 1172 tablet_bootstrap.cc:492] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:00.326247 1172 tablet_bootstrap.cc:654] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.332428 1181 raft_consensus.cc:740] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.332683 1181 consensus_queue.cc:260] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.334425 1181 ts_tablet_manager.cc:1434] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20260505 07:54:00.334683 1181 tablet_bootstrap.cc:492] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:54:00.336691 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7235a3ccef904913bbd67c403aac3a87 has no consensus state
I20260505 07:54:00.337062 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_38d26c09769b42159b828ec5bb429075"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.337498 599 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_a24821d9d3f14fee9ce992364d12721e" } new_table_name: "default.Table_4dc51be442ed4881b005509eb0911abf" modify_external_catalogs: true
W20260505 07:54:00.337648 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_38d26c09769b42159b828ec5bb429075 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.340608 594 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7219f9896fd94187ae714635077110af (table default.Table_ad09d6b42bc344c6ba96f4322d75601f [id=d82da77797e14074b53f00e8e741744c]). Expected version 1 got 0
I20260505 07:54:00.340816 594 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bd581a6799a14a8dafb80d411e8936f3 (table default.Table_f582c0d06a754ba7ad756b60cfde5174 [id=76b0eb99a1f343829e5ba35de6302cf2]). Expected version 1 got 0
I20260505 07:54:00.341925 594 master_service.cc:954] ReplaceTablet: received request to replace tablet d77ff90ec2ff4b6eb1795d149459c23a from {username='slave'} at 127.0.0.1:47238
I20260505 07:54:00.345870 1178 ts_tablet_manager.cc:1929] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.346062 1178 log.cc:1199] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/2ec347e754ab4b7282e68b15a9bbf993
I20260505 07:54:00.346455 1178 ts_tablet_manager.cc:1950] T 2ec347e754ab4b7282e68b15a9bbf993 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
W20260505 07:54:00.347060 598 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d77ff90ec2ff4b6eb1795d149459c23a consensus state has no leader
I20260505 07:54:00.347205 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 251) took 2699 ms. Trace:
I20260505 07:54:00.347308 1178 rpcz_store.cc:276] 0505 07:53:57.647398 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.718019 (+ 70621us) service_pool.cc:224] Handling call
0505 07:54:00.212068 (+2494049us) ts_tablet_manager.cc:1151] Deleting tablet 2ec347e754ab4b7282e68b15a9bbf993
0505 07:54:00.212082 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.313999 (+101917us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.345806 (+ 31807us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.347148 (+ 1342us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2473636}
I20260505 07:54:00.348035 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 267) took 2603 ms. Trace:
I20260505 07:54:00.348201 1178 rpcz_store.cc:276] 0505 07:53:57.744627 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.744680 (+ 53us) service_pool.cc:224] Handling call
0505 07:54:00.347886 (+2603206us) ts_tablet_manager.cc:1151] Deleting tablet f92c53c68a4b4e319e98b84128e8e6a4
0505 07:54:00.347894 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.347986 (+ 92us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2602903}
I20260505 07:54:00.349272 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 2ec347e754ab4b7282e68b15a9bbf993 (table default.Table_4d8c9530e5904e54ac3bdb2a82083180 [id=7e62bbf111c94a01b324b800bd6ced0b]) successfully deleted
I20260505 07:54:00.349356 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 266) took 2628 ms. Trace:
I20260505 07:54:00.349442 1178 rpcz_store.cc:276] 0505 07:53:57.720950 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.745576 (+ 24626us) service_pool.cc:224] Handling call
0505 07:54:00.349212 (+2603636us) ts_tablet_manager.cc:1151] Deleting tablet 397d33e908af4a0bb8dde26d273be01a
0505 07:54:00.349221 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.349304 (+ 83us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2603432}
I20260505 07:54:00.349772 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 265) took 2631 ms. Trace:
I20260505 07:54:00.349733 894 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac029d9a3c540f58f393e79bf7be30e (DEFAULT_TABLE table=default.Table_f40f1996d8454947afd1ff280e2b5fc1 [id=541feb1163924c0ca3fb40a84ced3826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.350139 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac029d9a3c540f58f393e79bf7be30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.350327 1178 rpcz_store.cc:276] 0505 07:53:57.720438 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.741195 (+ 20757us) service_pool.cc:224] Handling call
0505 07:54:00.349634 (+2608439us) ts_tablet_manager.cc:1151] Deleting tablet 323269e708964848a94c64e6a781a860
0505 07:54:00.349642 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.349726 (+ 84us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2591334}
I20260505 07:54:00.350696 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 268) took 2585 ms. Trace:
I20260505 07:54:00.350770 1178 rpcz_store.cc:276] 0505 07:53:57.765385 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.765501 (+ 116us) service_pool.cc:224] Handling call
0505 07:54:00.350543 (+2585042us) ts_tablet_manager.cc:1151] Deleting tablet d622612bdc404727a27b83769bf39738
0505 07:54:00.350551 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.350656 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2583687}
I20260505 07:54:00.351087 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 269) took 2536 ms. Trace:
I20260505 07:54:00.351159 1178 rpcz_store.cc:276] 0505 07:53:57.814197 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.815492 (+ 1295us) service_pool.cc:224] Handling call
0505 07:54:00.350963 (+2535471us) ts_tablet_manager.cc:1151] Deleting tablet 46274a44a06b4306b5dac43c66c85a80
0505 07:54:00.350970 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.351045 (+ 75us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2531636}
I20260505 07:54:00.351398 1178 tablet_replica.cc:333] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:00.351604 1178 raft_consensus.cc:2243] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.351775 1178 raft_consensus.cc:2272] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:00.351847 889 tablet_service.cc:1511] Processing CreateTablet for tablet 25072055f5534078adf5b5025614f951 (DEFAULT_TABLE table=default.Table_f93a81c5dfe740819f0f6ea5bf7f1067 [id=a8a6d42a6b6748d590ef5949a9d1e650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.352275 1178 ts_tablet_manager.cc:1916] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.353266 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 25072055f5534078adf5b5025614f951 (DEFAULT_TABLE table=default.Table_f93a81c5dfe740819f0f6ea5bf7f1067 [id=a8a6d42a6b6748d590ef5949a9d1e650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.353487 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 225c56ac920b448c89c38d6fb3c1bdef (DEFAULT_TABLE table=default.Table_ec95dd1d091844c3adb51935b2e48795 [id=8b6dda11eb884c7da54b226291f403c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.353619 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25072055f5534078adf5b5025614f951. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.353719 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f92c53c68a4b4e319e98b84128e8e6a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f92c53c68a4b4e319e98b84128e8e6a4 already in progress: creating tablet
I20260505 07:54:00.353852 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225c56ac920b448c89c38d6fb3c1bdef. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.353941 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 397d33e908af4a0bb8dde26d273be01a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 397d33e908af4a0bb8dde26d273be01a already in progress: creating tablet
W20260505 07:54:00.354115 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 323269e708964848a94c64e6a781a860 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 323269e708964848a94c64e6a781a860 already in progress: creating tablet
W20260505 07:54:00.354314 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet d622612bdc404727a27b83769bf39738 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d622612bdc404727a27b83769bf39738 already in progress: creating tablet
W20260505 07:54:00.354532 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 46274a44a06b4306b5dac43c66c85a80 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 46274a44a06b4306b5dac43c66c85a80 already in progress: creating tablet
I20260505 07:54:00.355085 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 6ca2852cee84476dbe5e5072a43087a1
W20260505 07:54:00.355279 599 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 08c6ed755fc64080a2310fe9a4e0a68b consensus state has no leader
I20260505 07:54:00.355458 596 catalog_manager.cc:6741] ReplaceTablet: tablet 6ca2852cee84476dbe5e5072a43087a1 deleted and replaced by tablet 01431ad112e8445487823efe8feb0dbe
I20260505 07:54:00.355911 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 02e06d8815034d91986cef12964860ea
I20260505 07:54:00.356353 600 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet b5a0cbf9c3d74cea92920189da332c1c
I20260505 07:54:00.356789 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet df1b62a30b2842b897a04c793bb5e7ce
I20260505 07:54:00.357126 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 01431ad112e8445487823efe8feb0dbe) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.357033 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 01431ad112e8445487823efe8feb0dbe) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.357440 1569 raft_consensus.cc:493] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.357612 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 1506cd31fe364e328867bbf70e36845e (DEFAULT_TABLE table=default.Table_ad8e6e942c73439fb64536b3ab1edc9c [id=c7cad0401e40485e9ed03160b9bed704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.357975 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1506cd31fe364e328867bbf70e36845e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.360215 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet df1b62a30b2842b897a04c793bb5e7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.360255 898 tablet_service.cc:1511] Processing CreateTablet for tablet 1506cd31fe364e328867bbf70e36845e (DEFAULT_TABLE table=default.Table_ad8e6e942c73439fb64536b3ab1edc9c [id=c7cad0401e40485e9ed03160b9bed704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.360361 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet d77ff90ec2ff4b6eb1795d149459c23a
I20260505 07:54:00.360080 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet b5a0cbf9c3d74cea92920189da332c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.360558 891 tablet_service.cc:1511] Processing CreateTablet for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (DEFAULT_TABLE table=default.Table_8f7c0f8991b1415abce7b31407477b52 [id=d6192f5cec174951bf17af2e65d01f79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.360909 594 catalog_manager.cc:6741] ReplaceTablet: tablet d77ff90ec2ff4b6eb1795d149459c23a deleted and replaced by tablet e5eaa91ecc1f43d18b313b6be4cc211e
I20260505 07:54:00.361487 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet d77ff90ec2ff4b6eb1795d149459c23a with delete_type TABLET_DATA_DELETED (replaced by tablet e5eaa91ecc1f43d18b313b6be4cc211e) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.364111 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 02e06d8815034d91986cef12964860ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.364637 895 tablet_service.cc:1558] Processing DeleteTablet for tablet df1b62a30b2842b897a04c793bb5e7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.364780 884 tablet_service.cc:1558] Processing DeleteTablet for tablet 02e06d8815034d91986cef12964860ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.364718 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac029d9a3c540f58f393e79bf7be30e (DEFAULT_TABLE table=default.Table_f40f1996d8454947afd1ff280e2b5fc1 [id=541feb1163924c0ca3fb40a84ced3826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.364903 901 tablet_service.cc:1511] Processing CreateTablet for tablet 225c56ac920b448c89c38d6fb3c1bdef (DEFAULT_TABLE table=default.Table_ec95dd1d091844c3adb51935b2e48795 [id=8b6dda11eb884c7da54b226291f403c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.365067 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac029d9a3c540f58f393e79bf7be30e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.365181 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25072055f5534078adf5b5025614f951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.366065 885 tablet_service.cc:1558] Processing DeleteTablet for tablet b5a0cbf9c3d74cea92920189da332c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.365674 1569 raft_consensus.cc:515] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.366832 896 tablet_service.cc:1558] Processing DeleteTablet for tablet d77ff90ec2ff4b6eb1795d149459c23a with delete_type TABLET_DATA_DELETED (replaced by tablet e5eaa91ecc1f43d18b313b6be4cc211e) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.366979 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225c56ac920b448c89c38d6fb3c1bdef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.367048 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet abe5f6f0774545b296bf1b03a08e145d
I20260505 07:54:00.367910 1569 leader_election.cc:290] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.368129 601 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet e010f23c4df647ab863e03c3cfed7898
I20260505 07:54:00.368330 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92c53c68a4b4e319e98b84128e8e6a4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:00.368599 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.368830 601 catalog_manager.cc:6741] ReplaceTablet: tablet e010f23c4df647ab863e03c3cfed7898 deleted and replaced by tablet 09019fdf60374b11bf3427f21a55b5e0
W20260505 07:54:00.369249 854 leader_election.cc:343] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.369629 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1506cd31fe364e328867bbf70e36845e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.370076 760 tablet_service.cc:1511] Processing CreateTablet for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (DEFAULT_TABLE table=default.Table_8f7c0f8991b1415abce7b31407477b52 [id=d6192f5cec174951bf17af2e65d01f79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.370440 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.372784 884 tablet_service.cc:1511] Processing CreateTablet for tablet 7235a3ccef904913bbd67c403aac3a87 (DEFAULT_TABLE table=default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.372985 896 tablet_service.cc:1558] Processing DeleteTablet for tablet abe5f6f0774545b296bf1b03a08e145d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.373590 895 tablet_service.cc:1511] Processing CreateTablet for tablet 0e08cf03ea0b495a94c6566ea6c8caad (DEFAULT_TABLE table=default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.373742 885 tablet_service.cc:1558] Processing DeleteTablet for tablet e010f23c4df647ab863e03c3cfed7898 with delete_type TABLET_DATA_DELETED (replaced by tablet 09019fdf60374b11bf3427f21a55b5e0) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.377318 1027 tablet_service.cc:1511] Processing CreateTablet for tablet e243711164f04faabbde53b0a09d90b1 (DEFAULT_TABLE table=default.Table_9c422539fda7423c820e2450a3f7c8db [id=5993b31e5350440f9715f1e416b90eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.377674 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e243711164f04faabbde53b0a09d90b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.379729 1024 tablet_service.cc:1511] Processing CreateTablet for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (DEFAULT_TABLE table=default.Table_8f7c0f8991b1415abce7b31407477b52 [id=d6192f5cec174951bf17af2e65d01f79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.379808 885 tablet_service.cc:1511] Processing CreateTablet for tablet e243711164f04faabbde53b0a09d90b1 (DEFAULT_TABLE table=default.Table_9c422539fda7423c820e2450a3f7c8db [id=5993b31e5350440f9715f1e416b90eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.380137 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.380399 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 01431ad112e8445487823efe8feb0dbe) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.381484 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 02e06d8815034d91986cef12964860ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.382221 754 tablet_service.cc:1558] Processing DeleteTablet for tablet b5a0cbf9c3d74cea92920189da332c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.383934 751 tablet_service.cc:1558] Processing DeleteTablet for tablet df1b62a30b2842b897a04c793bb5e7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.384080 754 tablet_service.cc:1558] Processing DeleteTablet for tablet d77ff90ec2ff4b6eb1795d149459c23a with delete_type TABLET_DATA_DELETED (replaced by tablet e5eaa91ecc1f43d18b313b6be4cc211e) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.385274 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ed41728f809d4b0eb56670c1410d14ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.389789 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92c53c68a4b4e319e98b84128e8e6a4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:00.389791 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ed41728f809d4b0eb56670c1410d14ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.390273 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 7235a3ccef904913bbd67c403aac3a87 (DEFAULT_TABLE table=default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.390631 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7235a3ccef904913bbd67c403aac3a87. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.392086 856 leader_election.cc:343] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.392205 856 leader_election.cc:304] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.392728 1569 raft_consensus.cc:2749] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.393693 1022 tablet_service.cc:1511] Processing CreateTablet for tablet 0e08cf03ea0b495a94c6566ea6c8caad (DEFAULT_TABLE table=default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.394336 760 tablet_service.cc:1511] Processing CreateTablet for tablet 225c56ac920b448c89c38d6fb3c1bdef (DEFAULT_TABLE table=default.Table_ec95dd1d091844c3adb51935b2e48795 [id=8b6dda11eb884c7da54b226291f403c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.394750 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 225c56ac920b448c89c38d6fb3c1bdef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.396905 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e08cf03ea0b495a94c6566ea6c8caad. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.384928 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 6ca2852cee84476dbe5e5072a43087a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ca2852cee84476dbe5e5072a43087a1 already in progress: creating tablet
I20260505 07:54:00.398509 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7235a3ccef904913bbd67c403aac3a87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.403028 1569 raft_consensus.cc:493] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.403070 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e08cf03ea0b495a94c6566ea6c8caad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.403226 1569 raft_consensus.cc:515] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.403867 1569 leader_election.cc:290] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.404489 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a60c015136246a3b02f70254c0bb67f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:00.404995 854 leader_election.cc:343] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.406284 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a60c015136246a3b02f70254c0bb67f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:00.407785 1176 tablet_bootstrap.cc:492] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:00.407954 1176 ts_tablet_manager.cc:1403] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.120s user 0.003s sys 0.001s
W20260505 07:54:00.397774 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 02e06d8815034d91986cef12964860ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02e06d8815034d91986cef12964860ea already in progress: creating tablet
W20260505 07:54:00.408372 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet b5a0cbf9c3d74cea92920189da332c1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b5a0cbf9c3d74cea92920189da332c1c already in progress: creating tablet
W20260505 07:54:00.408617 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet df1b62a30b2842b897a04c793bb5e7ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df1b62a30b2842b897a04c793bb5e7ce already in progress: creating tablet
W20260505 07:54:00.408802 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet d77ff90ec2ff4b6eb1795d149459c23a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d77ff90ec2ff4b6eb1795d149459c23a already in progress: creating tablet
I20260505 07:54:00.408811 1176 raft_consensus.cc:359] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.408993 1176 raft_consensus.cc:385] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.409060 1176 raft_consensus.cc:740] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.409293 1176 consensus_queue.cc:260] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.409992 751 tablet_service.cc:1558] Processing DeleteTablet for tablet abe5f6f0774545b296bf1b03a08e145d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.410181 754 tablet_service.cc:1511] Processing CreateTablet for tablet 7235a3ccef904913bbd67c403aac3a87 (DEFAULT_TABLE table=default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.410709 1176 ts_tablet_manager.cc:1434] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:00.411127 1171 ts_tablet_manager.cc:1916] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:00.411747 856 leader_election.cc:343] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.411861 856 leader_election.cc:304] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.411973 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7235a3ccef904913bbd67c403aac3a87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.412351 1569 raft_consensus.cc:2749] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.416985 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e243711164f04faabbde53b0a09d90b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.422629 1181 tablet_bootstrap.cc:654] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.424551 1176 tablet_bootstrap.cc:492] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:00.429849 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet e010f23c4df647ab863e03c3cfed7898 with delete_type TABLET_DATA_DELETED (replaced by tablet 09019fdf60374b11bf3427f21a55b5e0) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.430939 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8ef2c24cce2148ecb1409ecaa29a8130" } new_table_name: "default.Table_db92d1d6790447508a5f3b466214a285" modify_external_catalogs: true
I20260505 07:54:00.432505 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_085f78183e1e4000a83ccbb0ed350340" } new_table_name: "default.Table_57b735b31fec4ba88439afddef823171" modify_external_catalogs: true
I20260505 07:54:00.434057 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 49d7b9f5b6f74f8ea070f8ea7e545df7 (DEFAULT_TABLE table=default.Table_876e4bf0b9054b8e836299103ac03ef7 [id=a3b569958e05423588c9949eb7fe34a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.434752 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49d7b9f5b6f74f8ea070f8ea7e545df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.435344 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_93d07f8983d94584a9c9700054d460e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.435860 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_93d07f8983d94584a9c9700054d460e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:00.436901 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet abe5f6f0774545b296bf1b03a08e145d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abe5f6f0774545b296bf1b03a08e145d already in progress: creating tablet
W20260505 07:54:00.442946 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e7887c3462f74740b314e3bec36e280d consensus state has no leader
I20260505 07:54:00.442996 1178 ts_tablet_manager.cc:1929] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.443634 1178 log.cc:1199] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/5dab02a546b944f799045723a515118f
W20260505 07:54:00.443842 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 196866us
I20260505 07:54:00.445164 1544 raft_consensus.cc:493] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.445403 1544 raft_consensus.cc:515] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.446169 1544 leader_election.cc:290] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:00.446866 988 leader_election.cc:343] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 3272555a8c814c0ebe61ca32d715212b
I20260505 07:54:00.451368 1178 ts_tablet_manager.cc:1950] T 5dab02a546b944f799045723a515118f P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:00.451382 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_948a14e895244fb893e72073a38d1e14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.452253 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_948a14e895244fb893e72073a38d1e14 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.452327 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 270) took 2608 ms. Trace:
I20260505 07:54:00.452458 1178 rpcz_store.cc:276] 0505 07:53:57.843934 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.847765 (+ 3831us) service_pool.cc:224] Handling call
0505 07:54:00.351332 (+2503567us) ts_tablet_manager.cc:1151] Deleting tablet 5dab02a546b944f799045723a515118f
0505 07:54:00.351340 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.430356 (+ 79016us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.442595 (+ 12239us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.452286 (+ 9691us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2503181}
I20260505 07:54:00.452862 1178 tablet_replica.cc:333] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:00.452975 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 5dab02a546b944f799045723a515118f (table default.Table_2a2802255e0743c18185473d0dcded48 [id=b2adaf7422c7403c8b331c0d33f0ad53]) successfully deleted
I20260505 07:54:00.453087 1178 raft_consensus.cc:2243] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.453300 1178 raft_consensus.cc:2272] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:00.453837 1178 ts_tablet_manager.cc:1916] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.453955 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet abe5f6f0774545b296bf1b03a08e145d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
W20260505 07:54:00.456552 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet de1a667b14fe494c8ee4be0f47a340be consensus state has no leader
I20260505 07:54:00.424923 751 tablet_service.cc:1511] Processing CreateTablet for tablet 0e08cf03ea0b495a94c6566ea6c8caad (DEFAULT_TABLE table=default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.466270 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e08cf03ea0b495a94c6566ea6c8caad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.426090 761 tablet_service.cc:1511] Processing CreateTablet for tablet e243711164f04faabbde53b0a09d90b1 (DEFAULT_TABLE table=default.Table_9c422539fda7423c820e2450a3f7c8db [id=5993b31e5350440f9715f1e416b90eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.431804 757 tablet_service.cc:1511] Processing CreateTablet for tablet 49d7b9f5b6f74f8ea070f8ea7e545df7 (DEFAULT_TABLE table=default.Table_876e4bf0b9054b8e836299103ac03ef7 [id=a3b569958e05423588c9949eb7fe34a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.467936 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e243711164f04faabbde53b0a09d90b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.468159 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49d7b9f5b6f74f8ea070f8ea7e545df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.433660 754 tablet_service.cc:1558] Processing DeleteTablet for tablet e010f23c4df647ab863e03c3cfed7898 with delete_type TABLET_DATA_DELETED (replaced by tablet 09019fdf60374b11bf3427f21a55b5e0) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.433965 756 tablet_service.cc:1511] Processing CreateTablet for tablet 25072055f5534078adf5b5025614f951 (DEFAULT_TABLE table=default.Table_f93a81c5dfe740819f0f6ea5bf7f1067 [id=a8a6d42a6b6748d590ef5949a9d1e650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.437752 758 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac029d9a3c540f58f393e79bf7be30e (DEFAULT_TABLE table=default.Table_f40f1996d8454947afd1ff280e2b5fc1 [id=541feb1163924c0ca3fb40a84ced3826]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.439226 755 tablet_service.cc:1511] Processing CreateTablet for tablet 1506cd31fe364e328867bbf70e36845e (DEFAULT_TABLE table=default.Table_ad8e6e942c73439fb64536b3ab1edc9c [id=c7cad0401e40485e9ed03160b9bed704]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.450214 746 tablet_service.cc:1558] Processing DeleteTablet for tablet faf1140fe05e4f9493563ab8464c582b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.470523 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1506cd31fe364e328867bbf70e36845e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.470793 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac029d9a3c540f58f393e79bf7be30e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.471863 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e010f23c4df647ab863e03c3cfed7898 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e010f23c4df647ab863e03c3cfed7898 already in progress: creating tablet
I20260505 07:54:00.472941 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 9862b87d74974dc49d7dc9049b2a694a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.481601 1175 tablet_replica.cc:333] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:00.481781 1175 raft_consensus.cc:2243] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.476393 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 310ef5aef7e44ec5a062a01d2a175a11 (DEFAULT_TABLE table=default.Table_1aa571f4a9f54c8db6250dadf285b36b [id=77518ca228af4a6698c5d908a0c76b50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.484117 1175 raft_consensus.cc:2272] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:00.484337 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25072055f5534078adf5b5025614f951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.484678 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 with delete_type TABLET_DATA_DELETED (replaced by tablet 66082a7a75e345639e7a268727922fbb) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.480919 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 68e608ac01ef4a88906e585bd7ee25ac (DEFAULT_TABLE table=default.Table_38d26c09769b42159b828ec5bb429075 [id=fc9aba6852ee42cc83697f86e0e69c5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.481276 1178 ts_tablet_manager.cc:1929] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.486980 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b0cf443e3044e56a26b0a8a61c9cf7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.487371 1031 tablet_service.cc:1511] Processing CreateTablet for tablet f840d5f58161468385d5c4f116b78c3d (DEFAULT_TABLE table=default.Table_2847b3b378404489a41b1c874e251a83 [id=2528290a3f144a57b3867c9e7ed299ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.487671 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e608ac01ef4a88906e585bd7ee25ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.487918 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 310ef5aef7e44ec5a062a01d2a175a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.488536 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 4a8e6019406845ebb29051432a96dfd1 (DEFAULT_TABLE table=default.Table_dedf48e6bbc941f6b329f367499612f2 [id=c4d8c3d8aa554154a7015e6ff3d7b38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.489046 1035 tablet_service.cc:1511] Processing CreateTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e (DEFAULT_TABLE table=default.Table_1ff5286907c6464c8addf47c70eef7ef [id=f608e93cbc64485297c6dc2609a6eb87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.489043 1022 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea1d3829ede4b5582a022b64265df7e (DEFAULT_TABLE table=default.Table_4ce294c5bd3f49699d1e194c53ff9c07 [id=0ca5072e233148099ba64cca861c144c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.491449 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f840d5f58161468385d5c4f116b78c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.491400 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet 3272555a8c814c0ebe61ca32d715212b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.491462 749 tablet_service.cc:1511] Processing CreateTablet for tablet 4a8e6019406845ebb29051432a96dfd1 (DEFAULT_TABLE table=default.Table_dedf48e6bbc941f6b329f367499612f2 [id=c4d8c3d8aa554154a7015e6ff3d7b38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.491724 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 01431ad112e8445487823efe8feb0dbe (DEFAULT_TABLE table=default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.491941 1026 tablet_service.cc:1511] Processing CreateTablet for tablet b0d966bcd7f1419cbd090afc185ae78a (DEFAULT_TABLE table=default.Table_cc718440246d4ddfb49faf4602910188 [id=3ae3bc457dcb4db1b5f4360cb9e5c81c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.492362 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5eaa91ecc1f43d18b313b6be4cc211e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.493167 1178 log.cc:1199] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/2782cf753bc44bf4a102dd785ea3c6b8
I20260505 07:54:00.495432 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 09019fdf60374b11bf3427f21a55b5e0 (DEFAULT_TABLE table=default.Table_866ca8833a3a4cb6a04c41d3b7b76c23 [id=09bd9815349347ab9b75c4242c354c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.496419 889 tablet_service.cc:1511] Processing CreateTablet for tablet 01431ad112e8445487823efe8feb0dbe (DEFAULT_TABLE table=default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.496470 901 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea1d3829ede4b5582a022b64265df7e (DEFAULT_TABLE table=default.Table_4ce294c5bd3f49699d1e194c53ff9c07 [id=0ca5072e233148099ba64cca861c144c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.496656 899 tablet_service.cc:1511] Processing CreateTablet for tablet 09019fdf60374b11bf3427f21a55b5e0 (DEFAULT_TABLE table=default.Table_866ca8833a3a4cb6a04c41d3b7b76c23 [id=09bd9815349347ab9b75c4242c354c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.498518 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 172ca66c49354f549fa55fe63abf9970 (DEFAULT_TABLE table=default.Table_8c264d1fb97d491bb94ba720eef63e01 [id=4ca8dc50d71c42c6adbdedeb932d3040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.499677 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d966bcd7f1419cbd090afc185ae78a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.499670 898 tablet_service.cc:1511] Processing CreateTablet for tablet 4a8e6019406845ebb29051432a96dfd1 (DEFAULT_TABLE table=default.Table_dedf48e6bbc941f6b329f367499612f2 [id=c4d8c3d8aa554154a7015e6ff3d7b38c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.500137 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 172ca66c49354f549fa55fe63abf9970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.500267 891 tablet_service.cc:1511] Processing CreateTablet for tablet 49d7b9f5b6f74f8ea070f8ea7e545df7 (DEFAULT_TABLE table=default.Table_876e4bf0b9054b8e836299103ac03ef7 [id=a3b569958e05423588c9949eb7fe34a6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:00.500777 991 leader_election.cc:343] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 25a74b65d5d6459293ab4461f4d34cf2
I20260505 07:54:00.501020 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571257bbb7e94315b60d84d6aa4917b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:00.501222 921 raft_consensus.cc:2468] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:00.501696 991 leader_election.cc:304] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.500751 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 59a3cd77808c4e858ea2ef9ed2a28c38 (DEFAULT_TABLE table=default.Table_a31277b1c1ca4863976cb9726e83f13a [id=d4aea2d5304d4113ab26a474454b05de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.504025 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01431ad112e8445487823efe8feb0dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.505152 593 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b00ce8f351ce43edb2431c6e621d329e" } modify_external_catalogs: true
I20260505 07:54:00.505199 884 tablet_service.cc:1511] Processing CreateTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e (DEFAULT_TABLE table=default.Table_1ff5286907c6464c8addf47c70eef7ef [id=f608e93cbc64485297c6dc2609a6eb87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.505334 593 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b00ce8f351ce43edb2431c6e621d329e" } modify_external_catalogs: true
I20260505 07:54:00.506027 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d17cd5686eb94c3b8993d4310f0d31f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.506089 887 tablet_service.cc:1511] Processing CreateTablet for tablet 172ca66c49354f549fa55fe63abf9970 (DEFAULT_TABLE table=default.Table_8c264d1fb97d491bb94ba720eef63e01 [id=4ca8dc50d71c42c6adbdedeb932d3040]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:00.506573 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d17cd5686eb94c3b8993d4310f0d31f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.507157 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_97ee3338c1344152a0d674d42c76d5be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.507699 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97ee3338c1344152a0d674d42c76d5be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.505806 882 tablet_service.cc:1511] Processing CreateTablet for tablet b0d966bcd7f1419cbd090afc185ae78a (DEFAULT_TABLE table=default.Table_cc718440246d4ddfb49faf4602910188 [id=3ae3bc457dcb4db1b5f4360cb9e5c81c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.509784 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_af7b8d3a4e6843348198094736404edb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.510205 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af7b8d3a4e6843348198094736404edb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.510375 760 tablet_service.cc:1511] Processing CreateTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e (DEFAULT_TABLE table=default.Table_1ff5286907c6464c8addf47c70eef7ef [id=f608e93cbc64485297c6dc2609a6eb87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.510774 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a8e6019406845ebb29051432a96dfd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.510974 1579 raft_consensus.cc:2749] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.511251 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5eaa91ecc1f43d18b313b6be4cc211e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.513477 894 tablet_service.cc:1511] Processing CreateTablet for tablet 310ef5aef7e44ec5a062a01d2a175a11 (DEFAULT_TABLE table=default.Table_1aa571f4a9f54c8db6250dadf285b36b [id=77518ca228af4a6698c5d908a0c76b50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.514941 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a8e6019406845ebb29051432a96dfd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.515753 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 172ca66c49354f549fa55fe63abf9970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.514680 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_b4480c296bea4019ba121d0cb071e008"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.517405 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09019fdf60374b11bf3427f21a55b5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.518162 895 tablet_service.cc:1511] Processing CreateTablet for tablet 68e608ac01ef4a88906e585bd7ee25ac (DEFAULT_TABLE table=default.Table_38d26c09769b42159b828ec5bb429075 [id=fc9aba6852ee42cc83697f86e0e69c5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.518998 886 tablet_service.cc:1511] Processing CreateTablet for tablet 59a3cd77808c4e858ea2ef9ed2a28c38 (DEFAULT_TABLE table=default.Table_a31277b1c1ca4863976cb9726e83f13a [id=d4aea2d5304d4113ab26a474454b05de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.519515 991 leader_election.cc:304] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:00.520221 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b4480c296bea4019ba121d0cb071e008 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.521325 1580 raft_consensus.cc:2804] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:00.522385 1580 raft_consensus.cc:493] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:00.522519 1580 raft_consensus.cc:3060] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:00.521301 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a3cd77808c4e858ea2ef9ed2a28c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.523150 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a8e6019406845ebb29051432a96dfd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.523650 896 tablet_service.cc:1511] Processing CreateTablet for tablet f840d5f58161468385d5c4f116b78c3d (DEFAULT_TABLE table=default.Table_2847b3b378404489a41b1c874e251a83 [id=2528290a3f144a57b3867c9e7ed299ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.524163 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d966bcd7f1419cbd090afc185ae78a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.524133 1178 ts_tablet_manager.cc:1950] T 2782cf753bc44bf4a102dd785ea3c6b8 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:00.525287 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a3cd77808c4e858ea2ef9ed2a28c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.525548 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea1d3829ede4b5582a022b64265df7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.526687 1181 tablet_bootstrap.cc:492] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
W20260505 07:54:00.526710 991 leader_election.cc:343] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 3272555a8c814c0ebe61ca32d715212b
I20260505 07:54:00.530022 991 leader_election.cc:304] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.530259 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e608ac01ef4a88906e585bd7ee25ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.530917 1175 ts_tablet_manager.cc:1916] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.529182 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_1e6b1ec4673c45e6b7b6c2696722d84b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.528793 1579 raft_consensus.cc:493] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:00.532732 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e6b1ec4673c45e6b7b6c2696722d84b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.533664 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01431ad112e8445487823efe8feb0dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.534390 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea1d3829ede4b5582a022b64265df7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.535130 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f840d5f58161468385d5c4f116b78c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.536453 593 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet e8bf49b787ef4de095448751464708a4
I20260505 07:54:00.540856 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5eaa91ecc1f43d18b313b6be4cc211e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.540882 897 tablet_service.cc:1558] Processing DeleteTablet for tablet e8bf49b787ef4de095448751464708a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.541096 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 310ef5aef7e44ec5a062a01d2a175a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.541728 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09019fdf60374b11bf3427f21a55b5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.542313 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49d7b9f5b6f74f8ea070f8ea7e545df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.545229 754 tablet_service.cc:1511] Processing CreateTablet for tablet 310ef5aef7e44ec5a062a01d2a175a11 (DEFAULT_TABLE table=default.Table_1aa571f4a9f54c8db6250dadf285b36b [id=77518ca228af4a6698c5d908a0c76b50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.547839 746 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea1d3829ede4b5582a022b64265df7e (DEFAULT_TABLE table=default.Table_4ce294c5bd3f49699d1e194c53ff9c07 [id=0ca5072e233148099ba64cca861c144c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.547178 744 tablet_service.cc:1511] Processing CreateTablet for tablet 01431ad112e8445487823efe8feb0dbe (DEFAULT_TABLE table=default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.549271 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 271) took 2684 ms. Trace:
I20260505 07:54:00.549436 1178 rpcz_store.cc:276] 0505 07:53:57.844639 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.891357 (+ 46718us) service_pool.cc:224] Handling call
0505 07:54:00.452795 (+2561438us) ts_tablet_manager.cc:1151] Deleting tablet 2782cf753bc44bf4a102dd785ea3c6b8
0505 07:54:00.452804 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.462811 (+ 10007us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.481193 (+ 18382us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.529165 (+ 47972us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2561164}
I20260505 07:54:00.552160 752 tablet_service.cc:1511] Processing CreateTablet for tablet f840d5f58161468385d5c4f116b78c3d (DEFAULT_TABLE table=default.Table_2847b3b378404489a41b1c874e251a83 [id=2528290a3f144a57b3867c9e7ed299ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.553546 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 310ef5aef7e44ec5a062a01d2a175a11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.554697 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01431ad112e8445487823efe8feb0dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.554983 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f840d5f58161468385d5c4f116b78c3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.558276 1181 ts_tablet_manager.cc:1403] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.192s user 0.006s sys 0.020s
I20260505 07:54:00.561615 751 tablet_service.cc:1511] Processing CreateTablet for tablet 172ca66c49354f549fa55fe63abf9970 (DEFAULT_TABLE table=default.Table_8c264d1fb97d491bb94ba720eef63e01 [id=4ca8dc50d71c42c6adbdedeb932d3040]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.561858 757 tablet_service.cc:1511] Processing CreateTablet for tablet b0d966bcd7f1419cbd090afc185ae78a (DEFAULT_TABLE table=default.Table_cc718440246d4ddfb49faf4602910188 [id=3ae3bc457dcb4db1b5f4360cb9e5c81c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.562171 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 2782cf753bc44bf4a102dd785ea3c6b8 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]) successfully deleted
I20260505 07:54:00.562192 762 tablet_service.cc:1511] Processing CreateTablet for tablet 68e608ac01ef4a88906e585bd7ee25ac (DEFAULT_TABLE table=default.Table_38d26c09769b42159b828ec5bb429075 [id=fc9aba6852ee42cc83697f86e0e69c5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.563248 750 tablet_service.cc:1511] Processing CreateTablet for tablet 59a3cd77808c4e858ea2ef9ed2a28c38 (DEFAULT_TABLE table=default.Table_a31277b1c1ca4863976cb9726e83f13a [id=d4aea2d5304d4113ab26a474454b05de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.565084 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 279) took 2567 ms. Trace:
I20260505 07:54:00.566205 1569 raft_consensus.cc:493] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.566823 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_3e9abeb1ce814ef2840df1ce519f1768" } new_table_name: "default.Table_66f28ba637f24448816a64c7d316edee" modify_external_catalogs: true
I20260505 07:54:00.567370 1171 ts_tablet_manager.cc:1929] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.568099 1569 raft_consensus.cc:515] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.568449 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_54f371d41b5e4a54b08bb82a834ac0af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.568780 1569 leader_election.cc:290] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:00.568943 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54f371d41b5e4a54b08bb82a834ac0af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.568946 1580 raft_consensus.cc:515] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.570053 1544 raft_consensus.cc:2749] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.570489 1580 leader_election.cc:290] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.570495 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet e8bf49b787ef4de095448751464708a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.570725 1178 rpcz_store.cc:276] 0505 07:53:57.982349 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.982403 (+ 54us) service_pool.cc:224] Handling call
0505 07:54:00.549719 (+2567316us) ts_tablet_manager.cc:1151] Deleting tablet 141665fcd59048bf926acd0287f1853e
0505 07:54:00.549730 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.549829 (+ 99us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2554940}
W20260505 07:54:00.571741 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 141665fcd59048bf926acd0287f1853e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 141665fcd59048bf926acd0287f1853e already in progress: creating tablet
I20260505 07:54:00.563692 1579 raft_consensus.cc:515] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.574533 1171 log.cc:1199] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/3d83e9d4243e409794ffe8a342bce6b3
I20260505 07:54:00.576288 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea1d3829ede4b5582a022b64265df7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.578037 1579 leader_election.cc:290] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.579533 743 tablet_service.cc:1558] Processing DeleteTablet for tablet e8bf49b787ef4de095448751464708a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.579612 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd581a6799a14a8dafb80d411e8936f3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:00.579809 761 tablet_service.cc:1511] Processing CreateTablet for tablet 09019fdf60374b11bf3427f21a55b5e0 (DEFAULT_TABLE table=default.Table_866ca8833a3a4cb6a04c41d3b7b76c23 [id=09bd9815349347ab9b75c4242c354c57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.580883 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_9809ca8cef5f449c851dba1b86ebb1c7" } new_table_name: "default.Table_e22a94851ca24c8997ab83e9509506d2" modify_external_catalogs: true
I20260505 07:54:00.581081 743 tablet_service.cc:1511] Processing CreateTablet for tablet 7071df211bd34b86a751142bae3271b2 (DEFAULT_TABLE table=default.Table_93d07f8983d94584a9c9700054d460e6 [id=1c2e08372ceb44e7945a4abb176da05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.583099 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571257bbb7e94315b60d84d6aa4917b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:00.582096 1181 raft_consensus.cc:359] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.584169 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 172ca66c49354f549fa55fe63abf9970. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.584415 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59a3cd77808c4e858ea2ef9ed2a28c38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.584671 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7071df211bd34b86a751142bae3271b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.585155 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68e608ac01ef4a88906e585bd7ee25ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.585369 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09019fdf60374b11bf3427f21a55b5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.586331 593 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4a5f2bded41d47d59bd0da9575fb955a" } new_table_name: "default.Table_d99f0c6575024655ab3e0c737bc16910" modify_external_catalogs: true
I20260505 07:54:00.586607 592 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_db40b13ba59549c6bae350382395bceb" } modify_external_catalogs: true
I20260505 07:54:00.586752 592 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_db40b13ba59549c6bae350382395bceb" } modify_external_catalogs: true
I20260505 07:54:00.587821 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ad09d6b42bc344c6ba96f4322d75601f" } modify_external_catalogs: true
I20260505 07:54:00.587980 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ad09d6b42bc344c6ba96f4322d75601f" } modify_external_catalogs: true
I20260505 07:54:00.588030 885 tablet_service.cc:1511] Processing CreateTablet for tablet 6efb07cb60f64dfead739a3ca1343df9 (DEFAULT_TABLE table=default.Table_ed41728f809d4b0eb56670c1410d14ac [id=6e6683c2d09e4532aa3a56246c56bfd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.588409 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6efb07cb60f64dfead739a3ca1343df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.589213 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_40605e9a72634999bc80f464976c5dac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.589493 1171 ts_tablet_manager.cc:1950] T 3d83e9d4243e409794ffe8a342bce6b3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:00.589475 1181 raft_consensus.cc:385] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.587848 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ad3e5b8653224a0c929e29e9daf47ff5" } modify_external_catalogs: true
I20260505 07:54:00.589572 1181 raft_consensus.cc:740] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.590204 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ad3e5b8653224a0c929e29e9daf47ff5" } modify_external_catalogs: true
I20260505 07:54:00.590281 1181 consensus_queue.cc:260] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.590806 1181 ts_tablet_manager.cc:1434] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.032s user 0.001s sys 0.001s
W20260505 07:54:00.590205 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_40605e9a72634999bc80f464976c5dac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.591840 1181 tablet_bootstrap.cc:492] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:00.592293 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "faf1140fe05e4f9493563ab8464c582b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:00.592536 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 281) took 2567 ms. Trace:
I20260505 07:54:00.593520 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0d966bcd7f1419cbd090afc185ae78a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.596328 991 leader_election.cc:343] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: faf1140fe05e4f9493563ab8464c582b
W20260505 07:54:00.596596 988 leader_election.cc:343] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.596688 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd581a6799a14a8dafb80d411e8936f3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:00.597263 988 leader_election.cc:343] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:54:00.597397 988 leader_election.cc:304] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.600152 1023 tablet_service.cc:1511] Processing CreateTablet for tablet d896e290167b4d0081e8e8b831c8ae2a (DEFAULT_TABLE table=default.Table_948a14e895244fb893e72073a38d1e14 [id=85d8a9438bef4555b0ada2eceffefc0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.600652 1171 tablet_replica.cc:333] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:00.600841 1171 raft_consensus.cc:2243] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:00.601568 854 leader_election.cc:343] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.601835 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d896e290167b4d0081e8e8b831c8ae2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.602268 1178 rpcz_store.cc:276] 0505 07:53:58.011428 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.011857 (+ 429us) service_pool.cc:224] Handling call
0505 07:54:00.570999 (+2559142us) ts_tablet_manager.cc:1151] Deleting tablet d3d11bd19f074bbab101448b45426b7a
0505 07:54:00.571010 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.579034 (+ 8024us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2551091}
W20260505 07:54:00.602783 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet d3d11bd19f074bbab101448b45426b7a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d3d11bd19f074bbab101448b45426b7a already in progress: creating tablet
W20260505 07:54:00.603793 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 8eb7eaff8837485a8ae3ccb44c60471d consensus state has no leader
I20260505 07:54:00.604184 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571257bbb7e94315b60d84d6aa4917b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:00.605769 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 3d83e9d4243e409794ffe8a342bce6b3 (table default.Table_563025047e4142d4807295cff836ea99 [id=5a7033e8ecc940e49794281e421153ea]) successfully deleted
I20260505 07:54:00.605835 920 raft_consensus.cc:3060] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:00.607388 1579 raft_consensus.cc:2749] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.610844 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ee5371f299e44106859f3663fe6052b7" } modify_external_catalogs: true
I20260505 07:54:00.611011 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ee5371f299e44106859f3663fe6052b7" } modify_external_catalogs: true
I20260505 07:54:00.614005 1175 ts_tablet_manager.cc:1929] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.614089 1175 log.cc:1199] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/faf1140fe05e4f9493563ab8464c582b
I20260505 07:54:00.614336 1175 ts_tablet_manager.cc:1950] T faf1140fe05e4f9493563ab8464c582b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:00.584056 759 tablet_service.cc:1511] Processing CreateTablet for tablet 6efb07cb60f64dfead739a3ca1343df9 (DEFAULT_TABLE table=default.Table_ed41728f809d4b0eb56670c1410d14ac [id=6e6683c2d09e4532aa3a56246c56bfd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.620285 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6efb07cb60f64dfead739a3ca1343df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.620450 747 tablet_service.cc:1511] Processing CreateTablet for tablet d896e290167b4d0081e8e8b831c8ae2a (DEFAULT_TABLE table=default.Table_948a14e895244fb893e72073a38d1e14 [id=85d8a9438bef4555b0ada2eceffefc0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.620920 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d896e290167b4d0081e8e8b831c8ae2a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.627671 856 leader_election.cc:343] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.627712 1171 raft_consensus.cc:2272] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:00.627828 856 leader_election.cc:304] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.629398 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet faf1140fe05e4f9493563ab8464c582b (table default.Table_da8a2a5c54bf40a784a8a01b7245b280 [id=2170bb22bfb747c5b4f24ba83810f89e]) successfully deleted
W20260505 07:54:00.629746 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e8bf49b787ef4de095448751464708a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8bf49b787ef4de095448751464708a4 already in progress: creating tablet
I20260505 07:54:00.634289 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet feeafd614b3e4281a8e45b950e699b63
I20260505 07:54:00.634882 592 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 27bacc97d7534284970f218cf68ac75b
W20260505 07:54:00.635529 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 90a13ca612f241fc97ea12124677a0b1 consensus state has no leader
I20260505 07:54:00.636668 1176 tablet_bootstrap.cc:654] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.638021 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 7219f9896fd94187ae714635077110af
W20260505 07:54:00.643926 593 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f106a6d4855148fe83b1e90ea9f27d39 consensus state has no leader
I20260505 07:54:00.645231 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 280) took 2614 ms. Trace:
I20260505 07:54:00.647307 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_87a7c81488f44fa3a209fb2590694e26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.647866 743 tablet_service.cc:1558] Processing DeleteTablet for tablet 27bacc97d7534284970f218cf68ac75b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:54:00.648612 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_87a7c81488f44fa3a209fb2590694e26 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.648782 596 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7c3d336ba1db41b9b26a2fe5475bbab0" } new_table_name: "default.Table_d6881b2f97064289b8b333a795afa4ed" modify_external_catalogs: true
I20260505 07:54:00.650240 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 7219f9896fd94187ae714635077110af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.651721 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 6efb07cb60f64dfead739a3ca1343df9 (DEFAULT_TABLE table=default.Table_ed41728f809d4b0eb56670c1410d14ac [id=6e6683c2d09e4532aa3a56246c56bfd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.645361 1178 rpcz_store.cc:276] 0505 07:53:57.992448 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:57.992615 (+ 167us) service_pool.cc:224] Handling call
0505 07:54:00.602528 (+2609913us) ts_tablet_manager.cc:1151] Deleting tablet a9494c3b6a99487596a030856b371b68
0505 07:54:00.602536 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.607264 (+ 4728us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2578855}
I20260505 07:54:00.655707 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet feeafd614b3e4281a8e45b950e699b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.655839 1033 tablet_service.cc:1558] Processing DeleteTablet for tablet 7219f9896fd94187ae714635077110af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.655726 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 27bacc97d7534284970f218cf68ac75b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
W20260505 07:54:00.658419 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet a9494c3b6a99487596a030856b371b68 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9494c3b6a99487596a030856b371b68 already in progress: creating tablet
I20260505 07:54:00.658480 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 60772a9697ba417b879dc6025eeb8eec
I20260505 07:54:00.658795 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6ddd9e0382d845d1b89edce006361e1a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.659332 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ddd9e0382d845d1b89edce006361e1a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.663975 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_9cfdbaa4e1834e3fad8d09804f626d78"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.663954 594 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_866ca8833a3a4cb6a04c41d3b7b76c23" } new_table_name: "default.Table_5fcf5ea766444ab5b4515e76b4db6cfd" modify_external_catalogs: true
I20260505 07:54:00.665913 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 7071df211bd34b86a751142bae3271b2 (DEFAULT_TABLE table=default.Table_93d07f8983d94584a9c9700054d460e6 [id=1c2e08372ceb44e7945a4abb176da05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.667655 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6efb07cb60f64dfead739a3ca1343df9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.667650 1569 raft_consensus.cc:2749] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.667814 1176 tablet_bootstrap.cc:492] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:00.667950 1176 ts_tablet_manager.cc:1403] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.244s user 0.003s sys 0.001s
I20260505 07:54:00.668813 1176 raft_consensus.cc:359] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.668983 1176 raft_consensus.cc:385] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.669075 1176 raft_consensus.cc:740] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.669262 1176 consensus_queue.cc:260] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.669672 1176 ts_tablet_manager.cc:1434] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:00.670049 1176 tablet_bootstrap.cc:492] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:00.671828 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 60772a9697ba417b879dc6025eeb8eec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.673300 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 282) took 2654 ms. Trace:
I20260505 07:54:00.673418 1178 rpcz_store.cc:276] 0505 07:53:58.011634 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.011858 (+ 224us) service_pool.cc:224] Handling call
0505 07:54:00.657682 (+2645824us) ts_tablet_manager.cc:1151] Deleting tablet bc9ae6404dd64bd8bf4898768e94e4f1
0505 07:54:00.664433 (+ 6751us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.665689 (+ 1256us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2596372}
I20260505 07:54:00.675042 1176 tablet_bootstrap.cc:654] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.676481 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 289) took 2614 ms. Trace:
I20260505 07:54:00.676571 1178 rpcz_store.cc:276] 0505 07:53:58.060333 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.060931 (+ 598us) service_pool.cc:224] Handling call
0505 07:54:00.673669 (+2612738us) ts_tablet_manager.cc:1151] Deleting tablet 909306e1e5f34b7c8fbc0d86cafce531
0505 07:54:00.673680 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.674294 (+ 614us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2600422}
I20260505 07:54:00.677626 1178 tablet_replica.cc:333] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:00.677659 1579 raft_consensus.cc:493] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.678133 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7071df211bd34b86a751142bae3271b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.678278 1171 ts_tablet_manager.cc:1916] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.684090 1579 raft_consensus.cc:515] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.684764 1579 leader_election.cc:290] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.686499 1178 raft_consensus.cc:2243] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.686542 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ac06bb9d273457ead843f5dd754539c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:00.686846 762 tablet_service.cc:1558] Processing DeleteTablet for tablet feeafd614b3e4281a8e45b950e699b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.680276 898 tablet_service.cc:1511] Processing CreateTablet for tablet 7071df211bd34b86a751142bae3271b2 (DEFAULT_TABLE table=default.Table_93d07f8983d94584a9c9700054d460e6 [id=1c2e08372ceb44e7945a4abb176da05e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.688333 1569 raft_consensus.cc:493] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.689658 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 60772a9697ba417b879dc6025eeb8eec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:54:00.691416 991 leader_election.cc:343] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 3ac06bb9d273457ead843f5dd754539c
I20260505 07:54:00.692575 1579 raft_consensus.cc:493] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.692715 1579 raft_consensus.cc:515] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:00.694733 988 leader_election.cc:343] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.694942 988 leader_election.cc:304] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.695150 1580 raft_consensus.cc:2749] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.695339 897 tablet_service.cc:1511] Processing CreateTablet for tablet d896e290167b4d0081e8e8b831c8ae2a (DEFAULT_TABLE table=default.Table_948a14e895244fb893e72073a38d1e14 [id=85d8a9438bef4555b0ada2eceffefc0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.697984 1176 tablet_bootstrap.cc:492] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:00.698378 1176 ts_tablet_manager.cc:1403] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.001s
I20260505 07:54:00.699714 883 tablet_service.cc:1558] Processing DeleteTablet for tablet 27bacc97d7534284970f218cf68ac75b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.699662 1569 raft_consensus.cc:515] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.700289 1569 leader_election.cc:290] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.700810 1176 raft_consensus.cc:359] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.701324 1176 raft_consensus.cc:385] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.701471 1176 raft_consensus.cc:740] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.702035 1176 consensus_queue.cc:260] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.704169 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "976359601ad74b149696dc7f555b550c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:00.704579 1176 ts_tablet_manager.cc:1434] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260505 07:54:00.705442 1176 tablet_bootstrap.cc:492] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:00.707093 901 tablet_service.cc:1558] Processing DeleteTablet for tablet feeafd614b3e4281a8e45b950e699b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
W20260505 07:54:00.707890 854 leader_election.cc:343] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.710570 1176 tablet_bootstrap.cc:654] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.710844 1512 raft_consensus.cc:493] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.711009 1512 raft_consensus.cc:515] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.711827 1512 leader_election.cc:290] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:00.713374 1178 raft_consensus.cc:2272] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:00.713965 1178 ts_tablet_manager.cc:1916] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.714025 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d896e290167b4d0081e8e8b831c8ae2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.715818 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7071df211bd34b86a751142bae3271b2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.716364 718 leader_election.cc:343] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: c6dcbebddcdc4ad9a788c01134e1101c
W20260505 07:54:00.717690 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9cfdbaa4e1834e3fad8d09804f626d78 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:00.719846 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 27bacc97d7534284970f218cf68ac75b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27bacc97d7534284970f218cf68ac75b already in progress: creating tablet
I20260505 07:54:00.719852 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcbebddcdc4ad9a788c01134e1101c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:00.720029 1055 raft_consensus.cc:2468] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:00.720444 717 leader_election.cc:304] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
W20260505 07:54:00.720119 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 7219f9896fd94187ae714635077110af because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7219f9896fd94187ae714635077110af already in progress: creating tablet
I20260505 07:54:00.721897 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 7219f9896fd94187ae714635077110af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.725194 1512 raft_consensus.cc:2804] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:00.725324 1512 raft_consensus.cc:493] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:00.725425 1512 raft_consensus.cc:3060] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:00.727631 1181 tablet_bootstrap.cc:654] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.730247 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b404e50ae092419c9396772b3b14eeaa" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:00.732075 882 tablet_service.cc:1558] Processing DeleteTablet for tablet 60772a9697ba417b879dc6025eeb8eec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.732834 1172 tablet_bootstrap.cc:492] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:00.733199 1580 raft_consensus.cc:493] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.733355 1580 raft_consensus.cc:515] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.733912 1176 tablet_bootstrap.cc:492] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:00.733915 1580 leader_election.cc:290] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.734110 1176 ts_tablet_manager.cc:1403] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.029s user 0.007s sys 0.000s
I20260505 07:54:00.734987 920 raft_consensus.cc:2468] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:00.735177 1176 raft_consensus.cc:359] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.735340 1176 raft_consensus.cc:385] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.735424 1176 raft_consensus.cc:740] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.735512 991 leader_election.cc:304] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.736276 1580 raft_consensus.cc:2804] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:00.736438 1580 raft_consensus.cc:697] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:00.736687 1580 consensus_queue.cc:237] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.739377 1172 ts_tablet_manager.cc:1403] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.410s user 0.025s sys 0.043s
I20260505 07:54:00.740371 1176 consensus_queue.cc:260] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:54:00.741750 991 leader_election.cc:343] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 9862b87d74974dc49d7dc9049b2a694a
I20260505 07:54:00.742961 1579 leader_election.cc:290] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:00.742960 1172 raft_consensus.cc:359] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.743089 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "976359601ad74b149696dc7f555b550c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:00.743142 1172 raft_consensus.cc:385] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260505 07:54:00.743959 988 leader_election.cc:343] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:00.744011 596 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet c837877946604ef29f95dac9dd449bf3 consensus state has no leader
W20260505 07:54:00.744207 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet bc9ae6404dd64bd8bf4898768e94e4f1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc9ae6404dd64bd8bf4898768e94e4f1 already in progress: creating tablet
I20260505 07:54:00.744804 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9862b87d74974dc49d7dc9049b2a694a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:00.744882 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet feeafd614b3e4281a8e45b950e699b63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet feeafd614b3e4281a8e45b950e699b63 already in progress: creating tablet
I20260505 07:54:00.745025 781 raft_consensus.cc:2468] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:00.745229 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 60772a9697ba417b879dc6025eeb8eec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60772a9697ba417b879dc6025eeb8eec already in progress: creating tablet
I20260505 07:54:00.745532 988 leader_election.cc:304] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
W20260505 07:54:00.746698 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 909306e1e5f34b7c8fbc0d86cafce531 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 909306e1e5f34b7c8fbc0d86cafce531 already in progress: creating tablet
I20260505 07:54:00.749423 746 tablet_service.cc:1558] Processing DeleteTablet for tablet c6dcbebddcdc4ad9a788c01134e1101c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.749730 1172 raft_consensus.cc:740] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:54:00.750154 856 leader_election.cc:343] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.750288 856 leader_election.cc:304] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.732331 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b404e50ae092419c9396772b3b14eeaa" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:00.751713 1175 tablet_replica.cc:333] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:00.752238 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1ff5286907c6464c8addf47c70eef7ef" } modify_external_catalogs: true
W20260505 07:54:00.752408 594 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09019fdf60374b11bf3427f21a55b5e0 consensus state has no leader
I20260505 07:54:00.752453 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1ff5286907c6464c8addf47c70eef7ef" } modify_external_catalogs: true
I20260505 07:54:00.752615 1569 raft_consensus.cc:2749] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.753824 921 raft_consensus.cc:2468] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:00.754168 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_5e1ab4e85d4b46d8a5d98434da77c9a8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.754698 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5e1ab4e85d4b46d8a5d98434da77c9a8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.755326 1172 consensus_queue.cc:260] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.755899 1172 ts_tablet_manager.cc:1434] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.016s user 0.002s sys 0.000s
I20260505 07:54:00.755909 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_b853695095d24255abdbd32d33476f81"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.755539 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_55fc26564d414d72992d4651acdbc6bb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.756206 1172 tablet_bootstrap.cc:492] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:00.756377 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55fc26564d414d72992d4651acdbc6bb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:00.756378 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b853695095d24255abdbd32d33476f81 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.759202 991 leader_election.cc:304] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.759625 1544 raft_consensus.cc:2804] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:00.760854 1544 raft_consensus.cc:493] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:00.760962 1544 raft_consensus.cc:3060] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:00.764827 1579 raft_consensus.cc:2804] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:00.767802 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_a930977d21a04e229868a60d643ca173"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.768301 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a930977d21a04e229868a60d643ca173 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.770150 1569 raft_consensus.cc:493] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.770339 1569 raft_consensus.cc:515] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.770991 1569 leader_election.cc:290] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.771867 889 tablet_service.cc:1558] Processing DeleteTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 with delete_type TABLET_DATA_DELETED (replaced by tablet 4af9def97a454f3f9b63d637db42c110) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.776747 895 tablet_service.cc:1511] Processing CreateTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 (DEFAULT_TABLE table=default.Table_b4480c296bea4019ba121d0cb071e008 [id=b1381967e14d404aa5ab2fd0e7b69eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.776938 882 tablet_service.cc:1511] Processing CreateTablet for tablet 27f00169aad9420b892100acb4256f92 (DEFAULT_TABLE table=default.Table_97ee3338c1344152a0d674d42c76d5be [id=908887d51e8949ac81cebbcb0c6ddb6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.778515 889 tablet_service.cc:1511] Processing CreateTablet for tablet 21b48208e68247e78c12f0fec7ba07dc (DEFAULT_TABLE table=default.Table_1e6b1ec4673c45e6b7b6c2696722d84b [id=5d0d6b47936a454c9a93cc8af031807b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.779408 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_a4147e267b5f45e78812bde816ae7c8d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.780063 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a4147e267b5f45e78812bde816ae7c8d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.784118 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909306e1e5f34b7c8fbc0d86cafce531" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:00.786478 1544 raft_consensus.cc:515] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:00.791541 1579 raft_consensus.cc:493] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260505 07:54:00.792551 856 leader_election.cc:343] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.793314 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b404e50ae092419c9396772b3b14eeaa" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:00.793501 921 raft_consensus.cc:3060] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:00.796101 1579 raft_consensus.cc:3060] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:00.796515 921 raft_consensus.cc:2468] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:00.797025 991 leader_election.cc:304] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:00.797425 1567 raft_consensus.cc:2804] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:00.799283 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet e5eaa91ecc1f43d18b313b6be4cc211e
I20260505 07:54:00.799872 1544 leader_election.cc:290] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:00.809803 596 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c6c7a2a777c24a65a8217641ac14941c" } new_table_name: "default.Table_fc4b6502e4974457b4123c5ac53d567c" modify_external_catalogs: true
I20260505 07:54:00.810364 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2f5b72f58a24db0b58c15b47227bc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.811039 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27f00169aad9420b892100acb4256f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.816116 595 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_194f3e5aa086410eb3bd64b872d2f8af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.816715 595 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_194f3e5aa086410eb3bd64b872d2f8af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:00.819793 596 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4b9963a0f1a54ee9a73486bb1801ee74 consensus state has no leader
I20260505 07:54:00.820854 885 tablet_service.cc:1558] Processing DeleteTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.820998 1579 raft_consensus.cc:515] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.824810 1171 ts_tablet_manager.cc:1929] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260505 07:54:00.830186 991 leader_election.cc:343] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 9862b87d74974dc49d7dc9049b2a694a
I20260505 07:54:00.831224 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 21b48208e68247e78c12f0fec7ba07dc (DEFAULT_TABLE table=default.Table_1e6b1ec4673c45e6b7b6c2696722d84b [id=5d0d6b47936a454c9a93cc8af031807b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.831797 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b48208e68247e78c12f0fec7ba07dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.837023 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909306e1e5f34b7c8fbc0d86cafce531" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:00.837635 757 tablet_service.cc:1511] Processing CreateTablet for tablet 21b48208e68247e78c12f0fec7ba07dc (DEFAULT_TABLE table=default.Table_1e6b1ec4673c45e6b7b6c2696722d84b [id=5d0d6b47936a454c9a93cc8af031807b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.838160 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 9862b87d74974dc49d7dc9049b2a694a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.838368 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 9a06f77255544d7dabb71a4f70ada809 from {username='slave'} at 127.0.0.1:47238
I20260505 07:54:00.838507 758 tablet_service.cc:1558] Processing DeleteTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.838654 750 tablet_service.cc:1511] Processing CreateTablet for tablet 27f00169aad9420b892100acb4256f92 (DEFAULT_TABLE table=default.Table_97ee3338c1344152a0d674d42c76d5be [id=908887d51e8949ac81cebbcb0c6ddb6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.839156 751 tablet_service.cc:1511] Processing CreateTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 (DEFAULT_TABLE table=default.Table_b4480c296bea4019ba121d0cb071e008 [id=b1381967e14d404aa5ab2fd0e7b69eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.839267 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b48208e68247e78c12f0fec7ba07dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.841333 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b404e50ae092419c9396772b3b14eeaa" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:00.842338 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2f5b72f58a24db0b58c15b47227bc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.843384 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 9a06f77255544d7dabb71a4f70ada809
I20260505 07:54:00.843917 595 catalog_manager.cc:6741] ReplaceTablet: tablet 9a06f77255544d7dabb71a4f70ada809 deleted and replaced by tablet 12841efcef6041d89fc6d7d9b90448c0
I20260505 07:54:00.839128 1512 raft_consensus.cc:515] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:00.846138 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27f00169aad9420b892100acb4256f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.848374 1175 raft_consensus.cc:2243] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.849557 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_87a7c81488f44fa3a209fb2590694e26" } modify_external_catalogs: true
I20260505 07:54:00.849598 596 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c12be1928760438fab7c135df431bc79" } modify_external_catalogs: true
I20260505 07:54:00.849751 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_87a7c81488f44fa3a209fb2590694e26" } modify_external_catalogs: true
I20260505 07:54:00.849776 596 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_c12be1928760438fab7c135df431bc79" } modify_external_catalogs: true
W20260505 07:54:00.850069 587 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 8eb7eaff8837485a8ae3ccb44c60471d (table default.Table_66f28ba637f24448816a64c7d316edee [id=266ff796c7a74c23816dd9c422e9d005], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 8eb7eaff8837485a8ae3ccb44c60471d consensus state has no leader
I20260505 07:54:00.851055 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4dc51be442ed4881b005509eb0911abf" } new_table_name: "default.Table_7abf5cf3a7424664853b584a24a36437" modify_external_catalogs: true
I20260505 07:54:00.852151 597 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7001b868854949a5b9bee0ba45c2f522"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.852625 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_786f03e0f3404d69a9f8003f5e62368c" } new_table_name: "default.Table_dcd2d0ba71434f6bb31d92683f67408c" modify_external_catalogs: true
W20260505 07:54:00.852685 597 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7001b868854949a5b9bee0ba45c2f522 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.855067 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8091d8a01cbf47f3b270874cd5e6140e" } modify_external_catalogs: true
I20260505 07:54:00.855263 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_8091d8a01cbf47f3b270874cd5e6140e" } modify_external_catalogs: true
I20260505 07:54:00.858026 1171 log.cc:1199] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/8a60c015136246a3b02f70254c0bb67f
I20260505 07:54:00.858464 1171 ts_tablet_manager.cc:1950] T 8a60c015136246a3b02f70254c0bb67f P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:00.858798 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 381) took 1121 ms. Trace:
I20260505 07:54:00.858942 1171 rpcz_store.cc:276] 0505 07:53:59.737541 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.737595 (+ 54us) service_pool.cc:224] Handling call
0505 07:54:00.589886 (+852291us) ts_tablet_manager.cc:1151] Deleting tablet 8a60c015136246a3b02f70254c0bb67f
0505 07:54:00.589892 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.786073 (+196181us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.824725 (+ 38652us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.858769 (+ 34044us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":852116}
I20260505 07:54:00.827047 1172 tablet_bootstrap.cc:654] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.837453 1569 raft_consensus.cc:493] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.859427 1569 raft_consensus.cc:515] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.860034 1569 leader_election.cc:290] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:00.837687 854 leader_election.cc:343] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.860189 854 leader_election.cc:304] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.861459 1569 raft_consensus.cc:2749] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.841826 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b48208e68247e78c12f0fec7ba07dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.861611 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 8a60c015136246a3b02f70254c0bb67f (table default.Table_d9857ceb9b4d48dab8120e696f98e0e5 [id=65a99c8ba7b14ed0aec12382a2c8095c]) successfully deleted
W20260505 07:54:00.862042 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 17b4e886bebc4ab6a3c95290d18b4229 consensus state has no leader
W20260505 07:54:00.862386 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 08c6ed755fc64080a2310fe9a4e0a68b consensus state has no leader
I20260505 07:54:00.862722 594 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 4de395699508489ea8ac4e26b9438efd
I20260505 07:54:00.863332 596 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 09b46ff0101a4e529077b9e30492ace7
I20260505 07:54:00.846452 882 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a06f77255544d7dabb71a4f70ada809 with delete_type TABLET_DATA_DELETED (replaced by tablet 12841efcef6041d89fc6d7d9b90448c0) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.866153 595 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet 12841efcef6041d89fc6d7d9b90448c0
I20260505 07:54:00.866382 1176 ts_tablet_manager.cc:1434] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.106s user 0.003s sys 0.006s
I20260505 07:54:00.866791 1175 raft_consensus.cc:2272] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:00.867892 882 tablet_service.cc:1558] Processing DeleteTablet for tablet 09b46ff0101a4e529077b9e30492ace7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.869196 595 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 571257bbb7e94315b60d84d6aa4917b1 (table default.Table_4ab0ac01b8bf49ef96683dca41dfadd4 [id=6cb8bc8043474bad9f4278b7eef7c71a]). Expected version 3 got 0
I20260505 07:54:00.873843 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_575ab5b128ee4a1baebbe1f90baf005a" } modify_external_catalogs: true
I20260505 07:54:00.874046 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_575ab5b128ee4a1baebbe1f90baf005a" } modify_external_catalogs: true
I20260505 07:54:00.875517 595 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 571257bbb7e94315b60d84d6aa4917b1 (table default.Table_4ab0ac01b8bf49ef96683dca41dfadd4 [id=6cb8bc8043474bad9f4278b7eef7c71a]). Expected version 3 got 0
I20260505 07:54:00.828236 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a79578757824bfc9df129e3475e7af3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.823943 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 27f00169aad9420b892100acb4256f92 (DEFAULT_TABLE table=default.Table_97ee3338c1344152a0d674d42c76d5be [id=908887d51e8949ac81cebbcb0c6ddb6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.876726 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27f00169aad9420b892100acb4256f92. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:00.842144 988 leader_election.cc:343] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.877909 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7219f9896fd94187ae714635077110af" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:00.878507 856 leader_election.cc:343] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.854281 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.858870 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a06f77255544d7dabb71a4f70ada809 with delete_type TABLET_DATA_DELETED (replaced by tablet 12841efcef6041d89fc6d7d9b90448c0) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.861666 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcbebddcdc4ad9a788c01134e1101c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:00.878912 1055 raft_consensus.cc:3060] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:00.881703 1055 raft_consensus.cc:2468] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:00.881744 1569 raft_consensus.cc:493] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:00.881870 1569 raft_consensus.cc:515] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.882575 1569 leader_election.cc:290] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.882884 1023 tablet_service.cc:1511] Processing CreateTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 (DEFAULT_TABLE table=default.Table_b4480c296bea4019ba121d0cb071e008 [id=b1381967e14d404aa5ab2fd0e7b69eb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.857802 898 tablet_service.cc:1511] Processing CreateTablet for tablet 24f4854e8752493fb832ae2c12a033ce (DEFAULT_TABLE table=default.Table_af7b8d3a4e6843348198094736404edb [id=1a1c2d94983e409c8b75400de8adbae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.884042 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f4854e8752493fb832ae2c12a033ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.884042 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2f5b72f58a24db0b58c15b47227bc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.887738 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1838f89dc14a47f99f95f4cc5fdefd70" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:00.888316 856 leader_election.cc:343] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.890383 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 080b2984fcf2478a8def6540da63554d
I20260505 07:54:00.891237 1176 tablet_bootstrap.cc:492] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:00.891837 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 297537a92684425d8508f7b3b4c9d379 from {username='slave'} at 127.0.0.1:47238
I20260505 07:54:00.893502 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 080b2984fcf2478a8def6540da63554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.863338 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 24f4854e8752493fb832ae2c12a033ce (DEFAULT_TABLE table=default.Table_af7b8d3a4e6843348198094736404edb [id=1a1c2d94983e409c8b75400de8adbae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.865408 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 09b46ff0101a4e529077b9e30492ace7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:00.896466 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 382) took 1157 ms. Trace:
I20260505 07:54:00.896515 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f4854e8752493fb832ae2c12a033ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.899857 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a06f77255544d7dabb71a4f70ada809 with delete_type TABLET_DATA_DELETED (replaced by tablet 12841efcef6041d89fc6d7d9b90448c0) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.899878 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1838f89dc14a47f99f95f4cc5fdefd70" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:00.900393 854 leader_election.cc:343] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.900511 854 leader_election.cc:304] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.901113 1569 raft_consensus.cc:2749] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.902029 1579 leader_election.cc:290] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:00.912559 717 leader_election.cc:304] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:00.896587 1171 rpcz_store.cc:276] 0505 07:53:59.739001 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.739050 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:00.896291 (+1157241us) ts_tablet_manager.cc:1151] Deleting tablet 472441bd29704be59aabb0f7a4964bb0
0505 07:54:00.896300 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.896428 (+ 128us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1156989}
I20260505 07:54:00.913367 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 375) took 1238 ms. Trace:
W20260505 07:54:00.913688 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 472441bd29704be59aabb0f7a4964bb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 472441bd29704be59aabb0f7a4964bb0 already in progress: creating tablet
I20260505 07:54:00.914716 1178 ts_tablet_manager.cc:1929] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260505 07:54:00.915203 717 raft_consensus.cc:2686] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260505 07:54:00.914942 882 tablet_service.cc:1511] Processing CreateTablet for tablet ad5c570d4861460d8ab0c8ff8a7a3a03 (DEFAULT_TABLE table=default.Table_d17cd5686eb94c3b8993d4310f0d31f7 [id=8c0e89be595c4eb2b4d4103121cdccf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.915832 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad5c570d4861460d8ab0c8ff8a7a3a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.916379 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 09b46ff0101a4e529077b9e30492ace7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.916530 758 tablet_service.cc:1511] Processing CreateTablet for tablet 24f4854e8752493fb832ae2c12a033ce (DEFAULT_TABLE table=default.Table_af7b8d3a4e6843348198094736404edb [id=1a1c2d94983e409c8b75400de8adbae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.916188 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7219f9896fd94187ae714635077110af" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:00.917018 1512 leader_election.cc:290] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:00.918134 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 24f4854e8752493fb832ae2c12a033ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.918624 1175 ts_tablet_manager.cc:1916] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.922123 1171 rpcz_store.cc:276] 0505 07:53:59.675015 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.675793 (+ 778us) service_pool.cc:224] Handling call
0505 07:54:00.913239 (+1237446us) ts_tablet_manager.cc:1151] Deleting tablet f121fdd001ec4ea38ccd24a6afd9e51c
0505 07:54:00.913248 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.913336 (+ 88us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1169495}
I20260505 07:54:00.924976 1175 ts_tablet_manager.cc:1929] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:00.925110 1175 log.cc:1199] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/c6dcbebddcdc4ad9a788c01134e1101c
I20260505 07:54:00.925451 1175 ts_tablet_manager.cc:1950] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
W20260505 07:54:00.925534 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet f121fdd001ec4ea38ccd24a6afd9e51c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f121fdd001ec4ea38ccd24a6afd9e51c already in progress: creating tablet
I20260505 07:54:00.926050 1175 tablet_replica.cc:333] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:00.926209 1175 raft_consensus.cc:2243] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.926359 1175 raft_consensus.cc:2272] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:00.926905 854 leader_election.cc:343] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:00.927028 854 leader_election.cc:304] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:00.927253 1569 raft_consensus.cc:2749] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:00.927287 1175 ts_tablet_manager.cc:1916] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.927737 1171 tablet_replica.cc:333] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:00.927925 1171 raft_consensus.cc:2243] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.928076 1171 raft_consensus.cc:2272] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:00.928100 718 leader_election.cc:343] T c6dcbebddcdc4ad9a788c01134e1101c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: c6dcbebddcdc4ad9a788c01134e1101c
I20260505 07:54:00.928592 1171 ts_tablet_manager.cc:1916] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.928967 762 tablet_service.cc:1511] Processing CreateTablet for tablet ad5c570d4861460d8ab0c8ff8a7a3a03 (DEFAULT_TABLE table=default.Table_d17cd5686eb94c3b8993d4310f0d31f7 [id=8c0e89be595c4eb2b4d4103121cdccf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.929322 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad5c570d4861460d8ab0c8ff8a7a3a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.931879 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet c6dcbebddcdc4ad9a788c01134e1101c (table default.Table_ba04ccad67434c7884329b23efea4bfd [id=36cc8dc1683b4ef194ec299e4fed4de9]) successfully deleted
W20260505 07:54:00.932415 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a7fe9c7199cc4039ae877dc3d4aab17e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7fe9c7199cc4039ae877dc3d4aab17e already in progress: creating tablet
I20260505 07:54:00.940285 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 080b2984fcf2478a8def6540da63554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.950759 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_06ea1b3a616b4c5aa573a1c654538879"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.951305 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06ea1b3a616b4c5aa573a1c654538879 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.952929 595 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_f582c0d06a754ba7ad756b60cfde5174" } new_table_name: "default.Table_822258eaa3b14528a8bfa97e3e9d4d7a" modify_external_catalogs: true
I20260505 07:54:00.953974 597 master_service.cc:954] ReplaceTablet: received request to replace tablet aab3850799254dda9716fd5bf9404159 from {username='slave'} at 127.0.0.1:47238
I20260505 07:54:00.923195 1027 tablet_service.cc:1511] Processing CreateTablet for tablet ad5c570d4861460d8ab0c8ff8a7a3a03 (DEFAULT_TABLE table=default.Table_d17cd5686eb94c3b8993d4310f0d31f7 [id=8c0e89be595c4eb2b4d4103121cdccf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:00.956176 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad5c570d4861460d8ab0c8ff8a7a3a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:00.955566 1178 log.cc:1199] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/5de9e0fe8eb7477ab1cfa64b808575d8
I20260505 07:54:00.958288 1178 ts_tablet_manager.cc:1950] T 5de9e0fe8eb7477ab1cfa64b808575d8 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:00.959115 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 290) took 2850 ms. Trace:
I20260505 07:54:00.959337 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_1aa571f4a9f54c8db6250dadf285b36b" } new_table_name: "default.Table_823e01a26aa7450e869f0132a92ce378" modify_external_catalogs: true
I20260505 07:54:00.959296 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f8b3c6c6274f468fb9f56dad70e9043c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.959283 1178 rpcz_store.cc:276] 0505 07:53:58.108280 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.109947 (+ 1667us) service_pool.cc:224] Handling call
0505 07:54:00.676823 (+2566876us) ts_tablet_manager.cc:1151] Deleting tablet 5de9e0fe8eb7477ab1cfa64b808575d8
0505 07:54:00.676831 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.865696 (+188865us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.914632 (+ 48936us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.959075 (+ 44443us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2519425}
W20260505 07:54:00.960031 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f8b3c6c6274f468fb9f56dad70e9043c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.960261 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 291) took 2765 ms. Trace:
I20260505 07:54:00.960392 1178 rpcz_store.cc:276] 0505 07:53:58.195194 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.195840 (+ 646us) service_pool.cc:224] Handling call
0505 07:54:00.960090 (+2764250us) ts_tablet_manager.cc:1151] Deleting tablet ff7ed926a3ff4e7993864de734ca3c50
0505 07:54:00.960101 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.960221 (+ 120us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2759479}
I20260505 07:54:00.964437 1172 tablet_bootstrap.cc:492] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:00.964991 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 5de9e0fe8eb7477ab1cfa64b808575d8 (table default.Table_4011b7c55a4e4e149f8aa0b6f2981824 [id=b86b2850fe6f49b1907835ae468cf22e]) successfully deleted
W20260505 07:54:00.965237 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet ff7ed926a3ff4e7993864de734ca3c50 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ff7ed926a3ff4e7993864de734ca3c50 already in progress: creating tablet
I20260505 07:54:00.968974 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_043f3d0ae81c46a9a77b3264088fd1a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.969568 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_043f3d0ae81c46a9a77b3264088fd1a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.969945 600 catalog_manager.cc:5671] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:00.970324 600 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 571257bbb7e94315b60d84d6aa4917b1 (table default.Table_4ab0ac01b8bf49ef96683dca41dfadd4 [id=6cb8bc8043474bad9f4278b7eef7c71a]). Expected version 3 got 0
I20260505 07:54:00.972167 1567 raft_consensus.cc:697] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:00.972289 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_a11d31ae6a634b5693db8de72ca33cf8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:00.972457 1567 consensus_queue.cc:237] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:00.972693 601 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_d01b41eea75c46339079b05376f7d496"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.972821 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a11d31ae6a634b5693db8de72ca33cf8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:00.973196 601 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d01b41eea75c46339079b05376f7d496 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.974493 1178 tablet_replica.cc:333] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:00.974825 1172 ts_tablet_manager.cc:1403] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.219s user 0.004s sys 0.006s
I20260505 07:54:00.975183 599 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_7ed6d926da574429be8db03bde4b4221"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:00.975646 599 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ed6d926da574429be8db03bde4b4221 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:00.975616 1172 raft_consensus.cc:359] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.975811 1172 raft_consensus.cc:385] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:00.975869 1172 raft_consensus.cc:740] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:00.976085 1172 consensus_queue.cc:260] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:00.976752 1172 ts_tablet_manager.cc:1434] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:00.976929 1178 raft_consensus.cc:2243] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:00.977077 1178 raft_consensus.cc:2272] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:00.980806 595 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bd581a6799a14a8dafb80d411e8936f3 consensus state has no leader
I20260505 07:54:00.987680 1178 ts_tablet_manager.cc:1916] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:00.991575 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 080b2984fcf2478a8def6540da63554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:00.991714 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d6494590052459485059f5dce46f34f with delete_type TABLET_DATA_DELETED (replaced by tablet c822fafe72544ca99fde910c8e764755) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.992165 1172 tablet_bootstrap.cc:492] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:00.994107 1176 tablet_bootstrap.cc:654] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.994185 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet aab3850799254dda9716fd5bf9404159
I20260505 07:54:00.994673 597 catalog_manager.cc:6741] ReplaceTablet: tablet aab3850799254dda9716fd5bf9404159 deleted and replaced by tablet 5006965919824adbbb23d744fb9ff335
I20260505 07:54:00.995316 898 tablet_service.cc:1558] Processing DeleteTablet for tablet aab3850799254dda9716fd5bf9404159 with delete_type TABLET_DATA_DELETED (replaced by tablet 5006965919824adbbb23d744fb9ff335) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:00.995415 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9862b87d74974dc49d7dc9049b2a694a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:00.996978 1172 tablet_bootstrap.cc:654] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:00.997578 746 tablet_service.cc:1558] Processing DeleteTablet for tablet aab3850799254dda9716fd5bf9404159 with delete_type TABLET_DATA_DELETED (replaced by tablet 5006965919824adbbb23d744fb9ff335) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:01.000062 1171 ts_tablet_manager.cc:1929] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.000279 1171 log.cc:1199] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/f92c53c68a4b4e319e98b84128e8e6a4
I20260505 07:54:01.002561 1171 ts_tablet_manager.cc:1950] T f92c53c68a4b4e319e98b84128e8e6a4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.006222 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_a8605cf7bc3b44f39a895bfb7ad9934b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:01.006779 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8605cf7bc3b44f39a895bfb7ad9934b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:01.008363 597 master_service.cc:954] ReplaceTablet: received request to replace tablet 6546745e894545ff9c5f91bcc6e1a41e from {username='slave'} at 127.0.0.1:47238
I20260505 07:54:01.008388 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 387) took 1031 ms. Trace:
I20260505 07:54:01.010429 1171 rpcz_store.cc:276] 0505 07:53:59.976930 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.977337 (+ 407us) service_pool.cc:224] Handling call
0505 07:54:00.927670 (+950333us) ts_tablet_manager.cc:1151] Deleting tablet f92c53c68a4b4e319e98b84128e8e6a4
0505 07:54:00.927678 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:00.966321 (+ 38643us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:00.999978 (+ 33657us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.008336 (+ 8358us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":939908}
I20260505 07:54:01.010964 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b4480c296bea4019ba121d0cb071e008" } modify_external_catalogs: true
I20260505 07:54:01.011153 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b4480c296bea4019ba121d0cb071e008" } modify_external_catalogs: true
I20260505 07:54:01.011281 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 394) took 1023 ms. Trace:
I20260505 07:54:01.011376 1171 rpcz_store.cc:276] 0505 07:53:59.987345 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.987403 (+ 58us) service_pool.cc:224] Handling call
0505 07:54:01.011064 (+1023661us) ts_tablet_manager.cc:1151] Deleting tablet 0db162dfad2448e8ac428b3d3fbe17f7
0505 07:54:01.011078 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.011233 (+ 155us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1015562}
I20260505 07:54:01.011947 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 389) took 1029 ms. Trace:
I20260505 07:54:01.012027 1171 rpcz_store.cc:276] 0505 07:53:59.982825 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.983302 (+ 477us) service_pool.cc:224] Handling call
0505 07:54:01.011812 (+1028510us) ts_tablet_manager.cc:1151] Deleting tablet 9044763bed1a41739ecce4def4cca1fe
0505 07:54:01.011821 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.011900 (+ 79us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1015371}
I20260505 07:54:01.012362 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet f92c53c68a4b4e319e98b84128e8e6a4 (table default.Table_1496aa36a9c0478fa72f40d4595c2398 [id=73a1cc56dd244ba389866c21655a3a42]) successfully deleted
W20260505 07:54:01.012588 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 0db162dfad2448e8ac428b3d3fbe17f7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0db162dfad2448e8ac428b3d3fbe17f7 already in progress: creating tablet
W20260505 07:54:01.012820 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 9044763bed1a41739ecce4def4cca1fe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9044763bed1a41739ecce4def4cca1fe already in progress: creating tablet
I20260505 07:54:01.013058 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 395) took 1025 ms. Trace:
W20260505 07:54:01.016572 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 310ef5aef7e44ec5a062a01d2a175a11 consensus state has no leader
W20260505 07:54:01.018637 988 leader_election.cc:343] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:54:01.020248 1181 tablet_bootstrap.cc:492] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.020449 1181 ts_tablet_manager.cc:1403] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.429s user 0.002s sys 0.011s
I20260505 07:54:01.022850 1580 raft_consensus.cc:493] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.023525 592 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_ed3f6be0222a4436802966dd49d1dfad" } new_table_name: "default.Table_7e65acc795454f48ad6b8d0dc5dc163d" modify_external_catalogs: true
I20260505 07:54:01.024858 1580 raft_consensus.cc:515] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.025422 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet aab3850799254dda9716fd5bf9404159 with delete_type TABLET_DATA_DELETED (replaced by tablet 5006965919824adbbb23d744fb9ff335) from {username='slave'} at 127.0.0.1:45248
W20260505 07:54:01.026554 991 leader_election.cc:343] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 4a79578757824bfc9df129e3475e7af3
I20260505 07:54:01.024864 988 leader_election.cc:304] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.029088 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a79578757824bfc9df129e3475e7af3" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:01.029836 988 leader_election.cc:343] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.030231 1579 raft_consensus.cc:2749] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.030586 988 leader_election.cc:304] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.030880 1579 raft_consensus.cc:2749] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.031639 1596 raft_consensus.cc:493] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.032122 1580 leader_election.cc:290] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:01.031812 1596 raft_consensus.cc:515] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.034889 1596 leader_election.cc:290] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:01.036029 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_fcec571b3c7140669fe8d8c983cd616f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:01.036513 1176 tablet_bootstrap.cc:492] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
W20260505 07:54:01.036566 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fcec571b3c7140669fe8d8c983cd616f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:01.037889 1176 ts_tablet_manager.cc:1403] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.170s user 0.005s sys 0.001s
I20260505 07:54:01.039283 1171 rpcz_store.cc:276] 0505 07:53:59.987859 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.988246 (+ 387us) service_pool.cc:224] Handling call
0505 07:54:01.012918 (+1024672us) ts_tablet_manager.cc:1151] Deleting tablet f1058fcb9fa6457a85489049ca1d9ada
0505 07:54:01.012927 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.013002 (+ 75us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1015739}
I20260505 07:54:01.040024 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 396) took 1052 ms. Trace:
I20260505 07:54:01.040226 1171 rpcz_store.cc:276] 0505 07:53:59.987980 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.988515 (+ 535us) service_pool.cc:224] Handling call
0505 07:54:01.039834 (+1051319us) ts_tablet_manager.cc:1151] Deleting tablet 297537a92684425d8508f7b3b4c9d379
0505 07:54:01.039847 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.039961 (+ 114us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1042088}
W20260505 07:54:01.040448 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet f1058fcb9fa6457a85489049ca1d9ada because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f1058fcb9fa6457a85489049ca1d9ada already in progress: creating tablet
W20260505 07:54:01.040756 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 297537a92684425d8508f7b3b4c9d379 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 297537a92684425d8508f7b3b4c9d379 already in progress: creating tablet
I20260505 07:54:01.040949 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 393) took 1053 ms. Trace:
I20260505 07:54:01.041085 1171 rpcz_store.cc:276] 0505 07:53:59.987198 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.987469 (+ 271us) service_pool.cc:224] Handling call
0505 07:54:01.040727 (+1053258us) ts_tablet_manager.cc:1151] Deleting tablet 36b33d265ba04e93993c7e0a4efea1a4
0505 07:54:01.040738 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.040894 (+ 156us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1041238}
W20260505 07:54:01.041679 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 36b33d265ba04e93993c7e0a4efea1a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 36b33d265ba04e93993c7e0a4efea1a4 already in progress: creating tablet
I20260505 07:54:01.041975 1171 tablet_replica.cc:333] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:01.042254 1171 raft_consensus.cc:2243] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.042514 1171 raft_consensus.cc:2272] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.043207 1171 ts_tablet_manager.cc:1916] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.039530 1176 raft_consensus.cc:359] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.047976 1176 raft_consensus.cc:385] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.048548 1176 raft_consensus.cc:740] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.048950 1176 consensus_queue.cc:260] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.049413 1176 ts_tablet_manager.cc:1434] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260505 07:54:01.051656 1176 tablet_bootstrap.cc:492] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.054059 1175 ts_tablet_manager.cc:1929] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.054168 1175 log.cc:1199] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/9862b87d74974dc49d7dc9049b2a694a
I20260505 07:54:01.054543 1175 ts_tablet_manager.cc:1950] T 9862b87d74974dc49d7dc9049b2a694a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
W20260505 07:54:01.060111 991 leader_election.cc:343] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8b0cf443e3044e56a26b0a8a61c9cf7d
W20260505 07:54:01.061614 988 leader_election.cc:343] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 8b0cf443e3044e56a26b0a8a61c9cf7d
I20260505 07:54:01.061870 988 leader_election.cc:304] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.061275 1181 raft_consensus.cc:359] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.063220 1181 raft_consensus.cc:385] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.063350 1181 raft_consensus.cc:740] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.063638 1181 consensus_queue.cc:260] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.063963 1579 raft_consensus.cc:2749] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.064553 1181 ts_tablet_manager.cc:1434] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.044s user 0.003s sys 0.000s
I20260505 07:54:01.065076 1181 tablet_bootstrap.cc:492] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.066766 1181 tablet_bootstrap.cc:654] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.069569 1176 tablet_bootstrap.cc:654] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.069980 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ac06bb9d273457ead843f5dd754539c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:01.075230 594 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_49c6ff5135b14b6b8437bb753e1b6342" } modify_external_catalogs: true
I20260505 07:54:01.075448 594 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_49c6ff5135b14b6b8437bb753e1b6342" } modify_external_catalogs: true
I20260505 07:54:01.077350 600 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_822258eaa3b14528a8bfa97e3e9d4d7a" } modify_external_catalogs: true
I20260505 07:54:01.077536 600 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_822258eaa3b14528a8bfa97e3e9d4d7a" } modify_external_catalogs: true
I20260505 07:54:01.079133 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_7d1747ea770445058549830f424a7aa3" } new_table_name: "default.Table_304a4522acf84575bc1b2bdbf22a7ac4" modify_external_catalogs: true
I20260505 07:54:01.083873 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 9862b87d74974dc49d7dc9049b2a694a (table default.Table_d1ecbab8fbf047cf847ebbb4aeee4b63 [id=164a806dd1a9450ea55b4ea1be0caa64]) successfully deleted
W20260505 07:54:01.084117 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e5eaa91ecc1f43d18b313b6be4cc211e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5eaa91ecc1f43d18b313b6be4cc211e already in progress: creating tablet
W20260505 07:54:01.084542 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 09b46ff0101a4e529077b9e30492ace7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09b46ff0101a4e529077b9e30492ace7 already in progress: creating tablet
W20260505 07:54:01.084753 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 080b2984fcf2478a8def6540da63554d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 080b2984fcf2478a8def6540da63554d already in progress: creating tablet
I20260505 07:54:01.085071 1175 tablet_replica.cc:333] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:01.085264 1175 raft_consensus.cc:2243] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.085425 1175 raft_consensus.cc:2272] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:01.085559 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet aab3850799254dda9716fd5bf9404159 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aab3850799254dda9716fd5bf9404159 already in progress: creating tablet
I20260505 07:54:01.086151 1175 ts_tablet_manager.cc:1916] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.091240 593 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_3c9fe3aa59174b11a7f5f4488741deec" } new_table_name: "default.Table_92a11daebfd54782b9dbf6af7af4f214" modify_external_catalogs: true
I20260505 07:54:01.103644 1178 ts_tablet_manager.cc:1929] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.104748 1178 log.cc:1199] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/56cbbb89f6b84407b2c344ec62402b03
I20260505 07:54:01.105141 1178 ts_tablet_manager.cc:1950] T 56cbbb89f6b84407b2c344ec62402b03 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.105849 1176 tablet_bootstrap.cc:492] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.105935 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 292) took 2907 ms. Trace:
I20260505 07:54:01.106143 1176 ts_tablet_manager.cc:1403] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.056s user 0.002s sys 0.002s
I20260505 07:54:01.106045 1178 rpcz_store.cc:276] 0505 07:53:58.198676 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.198811 (+ 135us) service_pool.cc:224] Handling call
0505 07:54:00.974417 (+2775606us) ts_tablet_manager.cc:1151] Deleting tablet 56cbbb89f6b84407b2c344ec62402b03
0505 07:54:00.974429 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.016874 (+ 42445us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.023072 (+ 6198us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.105882 (+ 82810us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2773186}
I20260505 07:54:01.107157 1176 raft_consensus.cc:359] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:01.107446 1176 raft_consensus.cc:385] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.107615 1176 raft_consensus.cc:740] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.107909 1176 consensus_queue.cc:260] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:01.107861 599 catalog_manager.cc:5671] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:01.108467 1176 ts_tablet_manager.cc:1434] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:01.108939 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 293) took 2909 ms. Trace:
I20260505 07:54:01.109037 1178 rpcz_store.cc:276] 0505 07:53:58.201278 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.202248 (+ 970us) service_pool.cc:224] Handling call
0505 07:54:01.108738 (+2906490us) ts_tablet_manager.cc:1151] Deleting tablet bf76e868bbcb44b0b1887c84dd27f093
0505 07:54:01.108747 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.108894 (+ 147us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2899838}
I20260505 07:54:01.110463 599 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
W20260505 07:54:01.111034 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet bf76e868bbcb44b0b1887c84dd27f093 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf76e868bbcb44b0b1887c84dd27f093 already in progress: creating tablet
I20260505 07:54:01.111950 1176 tablet_bootstrap.cc:492] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.113776 1176 tablet_bootstrap.cc:654] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.118054 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 6429919aba17447dbdb393176360bbb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:01.118474 598 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_de416cf2bc944e20bb8b3d84e91585c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:01.119171 598 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de416cf2bc944e20bb8b3d84e91585c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:01.120512 1172 tablet_bootstrap.cc:492] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.120728 1172 ts_tablet_manager.cc:1403] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.129s user 0.003s sys 0.005s
I20260505 07:54:01.121601 1172 raft_consensus.cc:359] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.121846 1172 raft_consensus.cc:385] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.121984 1172 raft_consensus.cc:740] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.122299 1172 consensus_queue.cc:260] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.123153 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 304) took 2805 ms. Trace:
I20260505 07:54:01.123265 1178 rpcz_store.cc:276] 0505 07:53:58.318047 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.318097 (+ 50us) service_pool.cc:224] Handling call
0505 07:54:01.122980 (+2804883us) ts_tablet_manager.cc:1151] Deleting tablet 26f3544e74d941d286a2a0317b941ab6
0505 07:54:01.122991 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.123099 (+ 108us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2804681}
W20260505 07:54:01.123729 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 26f3544e74d941d286a2a0317b941ab6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26f3544e74d941d286a2a0317b941ab6 already in progress: creating tablet
I20260505 07:54:01.124150 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 298) took 2816 ms. Trace:
I20260505 07:54:01.124250 1178 rpcz_store.cc:276] 0505 07:53:58.307852 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.312848 (+ 4996us) service_pool.cc:224] Handling call
0505 07:54:01.123992 (+2811144us) ts_tablet_manager.cc:1151] Deleting tablet 5ad8a79245ad47738d079752e51aed2b
0505 07:54:01.124001 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.124097 (+ 96us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2800334}
I20260505 07:54:01.124516 1172 ts_tablet_manager.cc:1434] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
W20260505 07:54:01.124672 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 5ad8a79245ad47738d079752e51aed2b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5ad8a79245ad47738d079752e51aed2b already in progress: creating tablet
I20260505 07:54:01.125015 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 299) took 2817 ms. Trace:
I20260505 07:54:01.125101 1178 rpcz_store.cc:276] 0505 07:53:58.307950 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.313024 (+ 5074us) service_pool.cc:224] Handling call
0505 07:54:01.124891 (+2811867us) ts_tablet_manager.cc:1151] Deleting tablet 1602ccbae24544c4a6cf14ea073880b4
0505 07:54:01.124900 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.124971 (+ 71us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2797892}
W20260505 07:54:01.125463 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 1602ccbae24544c4a6cf14ea073880b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1602ccbae24544c4a6cf14ea073880b4 already in progress: creating tablet
I20260505 07:54:01.126660 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 308) took 2725 ms. Trace:
I20260505 07:54:01.126747 1178 rpcz_store.cc:276] 0505 07:53:58.401665 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.401723 (+ 58us) service_pool.cc:224] Handling call
0505 07:54:01.125672 (+2723949us) ts_tablet_manager.cc:1151] Deleting tablet a0f2f46d0e7840458cb992ce2a2dbea2
0505 07:54:01.125680 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.126611 (+ 931us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2534652}
I20260505 07:54:01.127175 1172 tablet_bootstrap.cc:492] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:01.127140 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet a0f2f46d0e7840458cb992ce2a2dbea2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a0f2f46d0e7840458cb992ce2a2dbea2 already in progress: creating tablet
I20260505 07:54:01.133947 1181 tablet_bootstrap.cc:492] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.134148 1181 ts_tablet_manager.cc:1403] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.069s user 0.002s sys 0.001s
I20260505 07:54:01.134991 1181 raft_consensus.cc:359] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.135243 1181 raft_consensus.cc:385] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.135326 1181 raft_consensus.cc:740] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.135625 1181 consensus_queue.cc:260] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.136229 1181 ts_tablet_manager.cc:1434] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:01.145478 597 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet 6546745e894545ff9c5f91bcc6e1a41e
I20260505 07:54:01.145987 597 catalog_manager.cc:6741] ReplaceTablet: tablet 6546745e894545ff9c5f91bcc6e1a41e deleted and replaced by tablet 9c4fc18f6faa488882d9bffb30793cdd
I20260505 07:54:01.147292 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c4fc18f6faa488882d9bffb30793cdd) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:01.147855 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c4fc18f6faa488882d9bffb30793cdd) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:01.148311 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c4fc18f6faa488882d9bffb30793cdd) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:01.149552 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 315) took 2555 ms. Trace:
I20260505 07:54:01.149619 1172 tablet_bootstrap.cc:654] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.149688 1178 rpcz_store.cc:276] 0505 07:53:58.590027 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.590546 (+ 519us) service_pool.cc:224] Handling call
0505 07:54:01.145127 (+2554581us) ts_tablet_manager.cc:1151] Deleting tablet c1a61c1d03d544c8aa01537ead97f83c
0505 07:54:01.145137 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.145216 (+ 79us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2540927}
I20260505 07:54:01.150355 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e707143a4e64725aa66db05b24999a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:01.150413 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 317) took 2477 ms. Trace:
I20260505 07:54:01.150487 1178 rpcz_store.cc:276] 0505 07:53:58.673013 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.677042 (+ 4029us) service_pool.cc:224] Handling call
0505 07:54:01.150261 (+2473219us) ts_tablet_manager.cc:1151] Deleting tablet f200abef13ac42698a9b96fc8af82ff0
0505 07:54:01.150271 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.150369 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2464384}
W20260505 07:54:01.150909 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f200abef13ac42698a9b96fc8af82ff0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f200abef13ac42698a9b96fc8af82ff0 already in progress: creating tablet
I20260505 07:54:01.151017 597 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_fa6246cc3ccd4d7082becde923f88828" } modify_external_catalogs: true
I20260505 07:54:01.151328 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 318) took 2371 ms. Trace:
I20260505 07:54:01.151410 1178 rpcz_store.cc:276] 0505 07:53:58.780082 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.781843 (+ 1761us) service_pool.cc:224] Handling call
0505 07:54:01.151178 (+2369335us) ts_tablet_manager.cc:1151] Deleting tablet ba616702dc6040bb805bb3ffc8a8a871
0505 07:54:01.151187 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.151288 (+ 101us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2369086}
I20260505 07:54:01.152971 597 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_fa6246cc3ccd4d7082becde923f88828" } modify_external_catalogs: true
W20260505 07:54:01.151862 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet ba616702dc6040bb805bb3ffc8a8a871 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba616702dc6040bb805bb3ffc8a8a871 already in progress: creating tablet
I20260505 07:54:01.154606 1171 ts_tablet_manager.cc:1929] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.154692 1178 tablet_replica.cc:333] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.154892 1178 raft_consensus.cc:2243] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.155082 1178 raft_consensus.cc:2272] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.156114 1181 tablet_bootstrap.cc:492] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.156313 1175 ts_tablet_manager.cc:1929] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.156468 1175 log.cc:1199] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/3ac06bb9d273457ead843f5dd754539c
I20260505 07:54:01.156869 1175 ts_tablet_manager.cc:1950] T 3ac06bb9d273457ead843f5dd754539c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:01.157850 1181 tablet_bootstrap.cc:654] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.157989 1178 ts_tablet_manager.cc:1916] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.158035 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 3ac06bb9d273457ead843f5dd754539c (table default.Table_4e9260f6bf4c4ea3b35c44a98dcf15f1 [id=4c5b3219388f43d4866a6ef3b32bf603]) successfully deleted
I20260505 07:54:01.159158 1175 tablet_replica.cc:333] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:01.159349 1175 raft_consensus.cc:2243] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.159536 1175 raft_consensus.cc:2272] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.160470 595 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet f2f5b72f58a24db0b58c15b47227bc60
I20260505 07:54:01.161814 1175 ts_tablet_manager.cc:1916] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.162578 758 tablet_service.cc:1558] Processing DeleteTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:01.163790 1176 tablet_bootstrap.cc:492] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.163955 1176 ts_tablet_manager.cc:1403] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.052s user 0.006s sys 0.003s
I20260505 07:54:01.164789 1176 raft_consensus.cc:359] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:01.164980 1176 raft_consensus.cc:385] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.165056 1176 raft_consensus.cc:740] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.165107 595 master_service.cc:954] ReplaceTablet: received request to replace tablet 46274a44a06b4306b5dac43c66c85a80 from {username='slave'} at 127.0.0.1:47238
I20260505 07:54:01.165308 1176 consensus_queue.cc:260] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:01.165594 595 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d" } modify_external_catalogs: true
I20260505 07:54:01.165781 595 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d" } modify_external_catalogs: true
I20260505 07:54:01.166407 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:01.167138 1176 ts_tablet_manager.cc:1434] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:01.167424 1176 tablet_bootstrap.cc:492] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.169665 1176 tablet_bootstrap.cc:654] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:01.177245 988 consensus_peers.cc:597] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e85b44337f62494c997c0008fc492aab. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:01.179857 592 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1175f7deec14fe39098ec89ab3bc049 consensus state has no leader
I20260505 07:54:01.180619 1567 raft_consensus.cc:493] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.180773 1567 raft_consensus.cc:515] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.181404 1567 leader_election.cc:290] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:01.182438 991 leader_election.cc:343] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 03f31efbc98c484c950c70d5e84d2136
I20260505 07:54:01.183974 1178 ts_tablet_manager.cc:1929] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.184094 1178 log.cc:1199] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/8573f8bd424641438e2aa6ff6869180e
I20260505 07:54:01.187978 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03f31efbc98c484c950c70d5e84d2136" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.194054 1175 ts_tablet_manager.cc:1929] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.161734 898 tablet_service.cc:1558] Processing DeleteTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:01.195874 1171 log.cc:1199] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/7585f00119e14006990ed551667b4486
I20260505 07:54:01.196412 1171 ts_tablet_manager.cc:1950] T 7585f00119e14006990ed551667b4486 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.197353 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 401) took 1160 ms. Trace:
I20260505 07:54:01.197546 1171 rpcz_store.cc:276] 0505 07:54:00.037325 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.037387 (+ 62us) service_pool.cc:224] Handling call
0505 07:54:01.041874 (+1004487us) ts_tablet_manager.cc:1151] Deleting tablet 7585f00119e14006990ed551667b4486
0505 07:54:01.041886 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.150829 (+108943us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.154516 (+ 3687us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.197299 (+ 42783us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1003493}
I20260505 07:54:01.198086 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 7585f00119e14006990ed551667b4486 (table default.Table_c61fca437ddb413e934595a23b52f40f [id=a14990bc69324c54a3c7e2b7d6116d11]) successfully deleted
I20260505 07:54:01.198459 1171 tablet_replica.cc:333] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:01.198714 1171 raft_consensus.cc:2243] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.198944 1171 raft_consensus.cc:2272] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.180744 1172 tablet_bootstrap.cc:492] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.199386 1172 ts_tablet_manager.cc:1403] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.072s user 0.004s sys 0.000s
I20260505 07:54:01.200209 1172 raft_consensus.cc:359] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.200484 1172 raft_consensus.cc:385] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.200625 1172 raft_consensus.cc:740] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.200995 1172 consensus_queue.cc:260] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.205101 1172 ts_tablet_manager.cc:1434] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
W20260505 07:54:01.188617 988 leader_election.cc:343] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.205485 1172 tablet_bootstrap.cc:492] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.205580 988 leader_election.cc:304] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.206120 1596 raft_consensus.cc:2749] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.207660 1178 ts_tablet_manager.cc:1950] T 8573f8bd424641438e2aa6ff6869180e P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.208803 1172 tablet_bootstrap.cc:654] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.211391 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_f99f927cdc404f66849fad37ee1f1445"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:01.213622 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f99f927cdc404f66849fad37ee1f1445 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:01.216486 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 319) took 2428 ms. Trace:
I20260505 07:54:01.216619 1178 rpcz_store.cc:276] 0505 07:53:58.788204 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.790168 (+ 1964us) service_pool.cc:224] Handling call
0505 07:54:01.154611 (+2364443us) ts_tablet_manager.cc:1151] Deleting tablet 8573f8bd424641438e2aa6ff6869180e
0505 07:54:01.154622 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.180625 (+ 26003us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.183892 (+ 3267us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.216436 (+ 32544us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2364088}
I20260505 07:54:01.216921 1181 tablet_bootstrap.cc:492] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.217056 1181 ts_tablet_manager.cc:1403] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.061s user 0.004s sys 0.013s
I20260505 07:54:01.217057 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 8573f8bd424641438e2aa6ff6869180e (table default.Table_7d1be5262b41414f8a0fc37fba1cc28a [id=128f12d4ca8c43f6887ae3240610ef60]) successfully deleted
I20260505 07:54:01.217346 1175 log.cc:1199] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/6429919aba17447dbdb393176360bbb0
I20260505 07:54:01.205137 1512 raft_consensus.cc:493] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.217732 1175 ts_tablet_manager.cc:1950] T 6429919aba17447dbdb393176360bbb0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:01.217614 1512 raft_consensus.cc:515] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.218377 1512 leader_election.cc:290] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.219039 1178 tablet_replica.cc:333] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.219399 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571257bbb7e94315b60d84d6aa4917b1" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:01.219480 1178 raft_consensus.cc:2243] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:01.219916 1178 raft_consensus.cc:2272] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.220290 1176 tablet_bootstrap.cc:492] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.220377 585 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 6429919aba17447dbdb393176360bbb0 (table default.Table_569c395ade0f4e2d8d0548b0d8e7cbbb [id=0aba4185373c484ba4f30d5efbb56746]) successfully deleted
I20260505 07:54:01.220422 1176 ts_tablet_manager.cc:1403] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.000s
I20260505 07:54:01.220485 1178 ts_tablet_manager.cc:1916] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:01.220628 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 6546745e894545ff9c5f91bcc6e1a41e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6546745e894545ff9c5f91bcc6e1a41e already in progress: creating tablet
W20260505 07:54:01.220868 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet f2f5b72f58a24db0b58c15b47227bc60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2f5b72f58a24db0b58c15b47227bc60 already in progress: creating tablet
I20260505 07:54:01.221259 1176 raft_consensus.cc:359] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.221450 1176 raft_consensus.cc:385] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.221544 1176 raft_consensus.cc:740] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.221777 1176 consensus_queue.cc:260] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.222236 1176 ts_tablet_manager.cc:1434] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:01.223106 1176 tablet_bootstrap.cc:492] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.224812 1176 tablet_bootstrap.cc:654] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.224848 594 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet b0b9e8eb767a4eceb583ff9b47000d58
I20260505 07:54:01.222779 1181 raft_consensus.cc:359] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.226620 1181 raft_consensus.cc:385] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.226761 1181 raft_consensus.cc:740] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.226758 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:01.227018 1181 consensus_queue.cc:260] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.228432 758 tablet_service.cc:1558] Processing DeleteTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:34194
I20260505 07:54:01.232504 1181 ts_tablet_manager.cc:1434] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.015s user 0.001s sys 0.002s
I20260505 07:54:01.212070 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 with delete_type TABLET_DATA_DELETED (replaced by tablet 42360f8ee4ff47c2af8d287f792207de) from {username='slave'} at 127.0.0.1:33752
W20260505 07:54:01.234318 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet b0b9e8eb767a4eceb583ff9b47000d58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0b9e8eb767a4eceb583ff9b47000d58 already in progress: creating tablet
I20260505 07:54:01.234725 594 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_abc41dd87e7a45c1b52bf036a9dfab27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:01.235250 594 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_abc41dd87e7a45c1b52bf036a9dfab27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:01.216778 1171 ts_tablet_manager.cc:1916] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.220707 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "571257bbb7e94315b60d84d6aa4917b1" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:01.235523 1181 tablet_bootstrap.cc:492] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.227049 895 tablet_service.cc:1558] Processing DeleteTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:01.236403 921 raft_consensus.cc:2393] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:01.236907 718 leader_election.cc:304] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:01.237320 1512 raft_consensus.cc:3060] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.240099 1512 raft_consensus.cc:2749] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.240899 1181 tablet_bootstrap.cc:654] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.243185 600 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Sending DeleteTablet for 3 replicas of tablet bd581a6799a14a8dafb80d411e8936f3
I20260505 07:54:01.245018 895 tablet_service.cc:1558] Processing DeleteTablet for tablet bd581a6799a14a8dafb80d411e8936f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:01.245476 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet bd581a6799a14a8dafb80d411e8936f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:01.246742 758 tablet_service.cc:1558] Processing DeleteTablet for tablet bd581a6799a14a8dafb80d411e8936f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:34194
W20260505 07:54:01.247309 585 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet bd581a6799a14a8dafb80d411e8936f3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bd581a6799a14a8dafb80d411e8936f3 already in progress: creating tablet
W20260505 07:54:01.247673 425 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 64359.1us
I20260505 07:54:01.249547 1178 ts_tablet_manager.cc:1929] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:01.249697 1178 log.cc:1199] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/3781d0bbcd1b4189bd836eeae4236f40
I20260505 07:54:01.250097 1178 ts_tablet_manager.cc:1950] T 3781d0bbcd1b4189bd836eeae4236f40 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.250954 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 320) took 2423 ms. Trace:
I20260505 07:54:01.251051 1178 rpcz_store.cc:276] 0505 07:53:58.827155 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.827215 (+ 60us) service_pool.cc:224] Handling call
0505 07:54:01.218978 (+2391763us) ts_tablet_manager.cc:1151] Deleting tablet 3781d0bbcd1b4189bd836eeae4236f40
0505 07:54:01.218987 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.225339 (+ 6352us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.249466 (+ 24127us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.250896 (+ 1430us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2391344}
I20260505 07:54:01.251518 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 3781d0bbcd1b4189bd836eeae4236f40 (table default.Table_6b95372e816b4dac9493d85379e26080 [id=b4934f65dfeb4a679111783301bb1fb9]) successfully deleted
I20260505 07:54:01.252640 600 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_2847b3b378404489a41b1c874e251a83" } new_table_name: "default.Table_89f2ae4a7f434e59a9039f98ec184e0b" modify_external_catalogs: true
I20260505 07:54:01.255100 1181 tablet_bootstrap.cc:492] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.255277 1181 ts_tablet_manager.cc:1403] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.001s
I20260505 07:54:01.256111 1181 raft_consensus.cc:359] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.256325 1181 raft_consensus.cc:385] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.256410 1181 raft_consensus.cc:740] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.256690 1181 consensus_queue.cc:260] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:01.259315 601 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6f207a3ffbc94b01abd9e9d6837f5c81 consensus state has no leader
I20260505 07:54:01.259634 601 master_service.cc:954] ReplaceTablet: received request to replace tablet 27bacc97d7534284970f218cf68ac75b from {username='slave'} at 127.0.0.1:47238
I20260505 07:54:01.261452 1178 tablet_replica.cc:333] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.261659 1178 raft_consensus.cc:2243] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.261833 1181 ts_tablet_manager.cc:1434] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:54:01.262316 1172 tablet_bootstrap.cc:492] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.262503 1172 ts_tablet_manager.cc:1403] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.057s user 0.003s sys 0.001s
I20260505 07:54:01.261857 1178 raft_consensus.cc:2272] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.263415 1172 raft_consensus.cc:359] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.263533 1178 ts_tablet_manager.cc:1916] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.264750 601 master_service.cc:954] ReplaceTablet: received request to replace tablet 60772a9697ba417b879dc6025eeb8eec from {username='slave'} at 127.0.0.1:47238
I20260505 07:54:01.265270 601 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:47238:
table { table_name: "default.Table_b853695095d24255abdbd32d33476f81" } new_table_name: "default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff" modify_external_catalogs: true
I20260505 07:54:01.266211 1181 tablet_bootstrap.cc:492] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.270145 1181 tablet_bootstrap.cc:654] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:01.273669 593 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9cbfb2345a1a4b218c2dddbae6667e47 consensus state has no leader
I20260505 07:54:01.277783 593 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_46550be844b4470e8239e2ba41399a68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:01.278369 593 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46550be844b4470e8239e2ba41399a68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:01.278632 1172 raft_consensus.cc:385] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.278718 1172 raft_consensus.cc:740] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.279018 1172 consensus_queue.cc:260] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.279759 1172 ts_tablet_manager.cc:1434] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.017s user 0.000s sys 0.002s
I20260505 07:54:01.282160 1172 tablet_bootstrap.cc:492] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.283396 1178 ts_tablet_manager.cc:1929] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.283507 1178 log.cc:1199] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/451e99a0dfec4823812e5991147b132e
I20260505 07:54:01.283965 1178 ts_tablet_manager.cc:1950] T 451e99a0dfec4823812e5991147b132e P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.285912 1176 tablet_bootstrap.cc:492] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.285771 1181 tablet_bootstrap.cc:492] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.286099 1176 ts_tablet_manager.cc:1403] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.063s user 0.003s sys 0.000s
I20260505 07:54:01.286237 1181 ts_tablet_manager.cc:1403] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.001s
I20260505 07:54:01.286918 1176 raft_consensus.cc:359] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.286965 1181 raft_consensus.cc:359] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.287137 1176 raft_consensus.cc:385] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.287209 1176 raft_consensus.cc:740] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.287313 1181 raft_consensus.cc:385] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.287451 1181 raft_consensus.cc:740] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.287477 1176 consensus_queue.cc:260] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.288069 1176 ts_tablet_manager.cc:1434] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260505 07:54:01.288892 1181 consensus_queue.cc:260] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.289466 1181 ts_tablet_manager.cc:1434] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:01.291308 1176 tablet_bootstrap.cc:492] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.291958 1181 tablet_bootstrap.cc:492] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.293264 1176 tablet_bootstrap.cc:654] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.293671 1181 tablet_bootstrap.cc:654] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.295073 592 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
I20260505 07:54:01.295217 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 321) took 2464 ms. Trace:
I20260505 07:54:01.278493 1171 ts_tablet_manager.cc:1929] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.295326 1178 rpcz_store.cc:276] 0505 07:53:58.830704 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.831452 (+ 748us) service_pool.cc:224] Handling call
0505 07:54:01.261354 (+2429902us) ts_tablet_manager.cc:1151] Deleting tablet 451e99a0dfec4823812e5991147b132e
0505 07:54:01.261365 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.276419 (+ 15054us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.283312 (+ 6893us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.295172 (+ 11860us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2429661}
I20260505 07:54:01.295607 1171 log.cc:1199] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/976359601ad74b149696dc7f555b550c
I20260505 07:54:01.295699 1178 tablet_replica.cc:333] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.295923 1178 raft_consensus.cc:2243] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.295987 1171 ts_tablet_manager.cc:1950] T 976359601ad74b149696dc7f555b550c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.296128 1178 raft_consensus.cc:2272] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.296658 1178 ts_tablet_manager.cc:1916] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.297659 1172 tablet_bootstrap.cc:654] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.302712 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 451e99a0dfec4823812e5991147b132e (table default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95]) successfully deleted
I20260505 07:54:01.303854 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 3781d0bbcd1b4189bd836eeae4236f40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:45248
I20260505 07:54:01.304268 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 411) took 1088 ms. Trace:
I20260505 07:54:01.304381 1171 rpcz_store.cc:276] 0505 07:54:00.216240 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.226815 (+ 10575us) service_pool.cc:224] Handling call
0505 07:54:01.198394 (+971579us) ts_tablet_manager.cc:1151] Deleting tablet 976359601ad74b149696dc7f555b550c
0505 07:54:01.198405 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.238767 (+ 40362us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.278412 (+ 39645us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.304218 (+ 25806us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":971364}
I20260505 07:54:01.304858 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 976359601ad74b149696dc7f555b550c (table default.Table_cb7435665bdd45ff8f517372a18d10b6 [id=0d5a336f16bb433fb2ee0f23189afc91]) successfully deleted
I20260505 07:54:01.307925 1171 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33752 (request call id 412) took 1009 ms. Trace:
I20260505 07:54:01.308109 1171 rpcz_store.cc:276] 0505 07:54:00.298462 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.298513 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:01.307723 (+1009210us) ts_tablet_manager.cc:1151] Deleting tablet e82e9ae417784b7e83f4843b129d0580
0505 07:54:01.307738 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.307866 (+ 128us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1008990}
W20260505 07:54:01.308655 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e82e9ae417784b7e83f4843b129d0580 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e82e9ae417784b7e83f4843b129d0580 already in progress: creating tablet
W20260505 07:54:01.308997 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 6ca2852cee84476dbe5e5072a43087a1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6ca2852cee84476dbe5e5072a43087a1 already in progress: creating tablet
W20260505 07:54:01.311101 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet df1b62a30b2842b897a04c793bb5e7ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df1b62a30b2842b897a04c793bb5e7ce already in progress: creating tablet
W20260505 07:54:01.311941 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 02e06d8815034d91986cef12964860ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02e06d8815034d91986cef12964860ea already in progress: creating tablet
W20260505 07:54:01.314175 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet b5a0cbf9c3d74cea92920189da332c1c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b5a0cbf9c3d74cea92920189da332c1c already in progress: creating tablet
I20260505 07:54:01.316100 592 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_ca2d7147756c4590a83c26ad0af48294"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:01.316697 592 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca2d7147756c4590a83c26ad0af48294 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:01.317925 1181 tablet_bootstrap.cc:492] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.317963 1176 tablet_bootstrap.cc:492] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.318089 1181 ts_tablet_manager.cc:1403] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.001s
I20260505 07:54:01.318112 1176 ts_tablet_manager.cc:1403] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.027s user 0.004s sys 0.000s
I20260505 07:54:01.318238 1171 tablet_replica.cc:333] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:01.318490 1171 raft_consensus.cc:2243] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.318921 1181 raft_consensus.cc:359] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.319164 1181 raft_consensus.cc:385] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.319085 1176 raft_consensus.cc:359] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.319244 1181 raft_consensus.cc:740] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.319293 1176 raft_consensus.cc:385] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.319401 1176 raft_consensus.cc:740] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.319535 1181 consensus_queue.cc:260] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.319664 1176 consensus_queue.cc:260] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:54:01.319892 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet abe5f6f0774545b296bf1b03a08e145d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abe5f6f0774545b296bf1b03a08e145d already in progress: creating tablet
W20260505 07:54:01.320111 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e010f23c4df647ab863e03c3cfed7898 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e010f23c4df647ab863e03c3cfed7898 already in progress: creating tablet
I20260505 07:54:01.320199 1176 ts_tablet_manager.cc:1434] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:01.320082 895 tablet_service.cc:1558] Processing DeleteTablet for tablet bf76e868bbcb44b0b1887c84dd27f093 with delete_type TABLET_DATA_DELETED (replaced by tablet c837877946604ef29f95dac9dd449bf3) from {username='slave'} at 127.0.0.1:33752
I20260505 07:54:01.319692 1171 raft_consensus.cc:2272] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.321238 1171 ts_tablet_manager.cc:1916] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.323572 1181 ts_tablet_manager.cc:1434] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:01.323922 1181 tablet_bootstrap.cc:492] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.324852 1172 tablet_bootstrap.cc:492] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.325062 1172 ts_tablet_manager.cc:1403] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.001s
I20260505 07:54:01.325910 1172 raft_consensus.cc:359] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.326190 1172 raft_consensus.cc:385] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.326311 1172 raft_consensus.cc:740] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.327013 1172 consensus_queue.cc:260] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.327458 1181 tablet_bootstrap.cc:654] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.329886 1172 ts_tablet_manager.cc:1434] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:54:01.330219 1172 tablet_bootstrap.cc:492] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.331030 1176 tablet_bootstrap.cc:492] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.333825 1178 ts_tablet_manager.cc:1929] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260505 07:54:01.334129 1176 tablet_bootstrap.cc:654] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.342726 1178 log.cc:1199] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/8ca4cf248adb4f768c5586d72614ad42
I20260505 07:54:01.343262 1178 ts_tablet_manager.cc:1950] T 8ca4cf248adb4f768c5586d72614ad42 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.344161 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 322) took 2392 ms. Trace:
I20260505 07:54:01.344626 1178 rpcz_store.cc:276] 0505 07:53:58.951295 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:58.951523 (+ 228us) service_pool.cc:224] Handling call
0505 07:54:01.295634 (+2344111us) ts_tablet_manager.cc:1151] Deleting tablet 8ca4cf248adb4f768c5586d72614ad42
0505 07:54:01.295644 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.316773 (+ 21129us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.333745 (+ 16972us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.344105 (+ 10360us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2330673}
I20260505 07:54:01.345335 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 8ca4cf248adb4f768c5586d72614ad42 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]) successfully deleted
I20260505 07:54:01.345883 1178 tablet_replica.cc:333] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.347563 1172 tablet_bootstrap.cc:654] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.348608 1178 raft_consensus.cc:2243] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.348821 1178 raft_consensus.cc:2272] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.349345 1178 ts_tablet_manager.cc:1916] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.351542 1176 tablet_bootstrap.cc:492] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.351750 1176 ts_tablet_manager.cc:1403] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.001s
I20260505 07:54:01.352587 1176 raft_consensus.cc:359] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.352818 1176 raft_consensus.cc:385] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.352901 1176 raft_consensus.cc:740] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.352867 596 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_6b6489b552be4fef93c6cace622b4253"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:01.353189 1176 consensus_queue.cc:260] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:01.353412 596 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b6489b552be4fef93c6cace622b4253 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:01.357324 1176 ts_tablet_manager.cc:1434] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260505 07:54:01.361409 1176 tablet_bootstrap.cc:492] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.363216 1176 tablet_bootstrap.cc:654] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.366334 1181 tablet_bootstrap.cc:492] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.366485 1181 ts_tablet_manager.cc:1403] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.000s
I20260505 07:54:01.367372 1181 raft_consensus.cc:359] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.367559 1181 raft_consensus.cc:385] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.367671 1181 raft_consensus.cc:740] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.367944 1181 consensus_queue.cc:260] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.368948 1181 ts_tablet_manager.cc:1434] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:01.370929 1181 tablet_bootstrap.cc:492] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.373798 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu pid 567
I20260505 07:54:01.375481 1178 ts_tablet_manager.cc:1929] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.375649 1178 log.cc:1199] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/c7652191b1d44ba59d856a2b0d7017b1
I20260505 07:54:01.376330 631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260505 07:54:01.378131 1181 tablet_bootstrap.cc:654] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:01.380743 600 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f840d5f58161468385d5c4f116b78c3d consensus state has no leader
I20260505 07:54:01.383548 1569 raft_consensus.cc:493] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.389325 1569 raft_consensus.cc:515] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.390203 1178 ts_tablet_manager.cc:1950] T c7652191b1d44ba59d856a2b0d7017b1 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.390653 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840eebf9e7404bf7807b7e51996957e5" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:01.391000 1181 tablet_bootstrap.cc:492] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.391192 1181 ts_tablet_manager.cc:1403] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.002s
I20260505 07:54:01.391168 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 334) took 2366 ms. Trace:
I20260505 07:54:01.391355 1178 rpcz_store.cc:276] 0505 07:53:59.024425 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.027622 (+ 3197us) service_pool.cc:224] Handling call
0505 07:54:01.345801 (+2318179us) ts_tablet_manager.cc:1151] Deleting tablet c7652191b1d44ba59d856a2b0d7017b1
0505 07:54:01.345811 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.367003 (+ 21192us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.375393 (+ 8390us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.391101 (+ 15708us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2317936}
I20260505 07:54:01.392050 1569 leader_election.cc:290] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:01.392215 856 leader_election.cc:343] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.392275 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 335) took 2294 ms. Trace:
I20260505 07:54:01.392532 1176 tablet_bootstrap.cc:492] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.392609 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840eebf9e7404bf7807b7e51996957e5" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.392695 1176 ts_tablet_manager.cc:1403] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.031s user 0.000s sys 0.003s
I20260505 07:54:01.393093 1178 rpcz_store.cc:276] 0505 07:53:59.098308 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.098436 (+ 128us) service_pool.cc:224] Handling call
0505 07:54:01.392056 (+2293620us) ts_tablet_manager.cc:1151] Deleting tablet 727cacd242844455915bd9963d2db19f
0505 07:54:01.392068 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.392240 (+ 172us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2290919}
I20260505 07:54:01.392551 1181 raft_consensus.cc:359] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:01.393375 854 leader_election.cc:343] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.393458 1181 raft_consensus.cc:385] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.393544 1181 raft_consensus.cc:740] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.393530 854 leader_election.cc:304] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.393800 1181 consensus_queue.cc:260] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.394083 1569 raft_consensus.cc:2749] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.394325 1181 ts_tablet_manager.cc:1434] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:01.394790 1176 raft_consensus.cc:359] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.394980 1176 raft_consensus.cc:385] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.395085 1176 raft_consensus.cc:740] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.395321 1176 consensus_queue.cc:260] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.398761 1176 ts_tablet_manager.cc:1434] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260505 07:54:01.399909 1181 tablet_bootstrap.cc:492] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.400678 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 336) took 2299 ms. Trace:
I20260505 07:54:01.400846 1178 rpcz_store.cc:276] 0505 07:53:59.101566 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.101820 (+ 254us) service_pool.cc:224] Handling call
0505 07:54:01.394471 (+2292651us) ts_tablet_manager.cc:1151] Deleting tablet 2d86b8b4dadf4fd2b6409a8cb70e313e
0505 07:54:01.400549 (+ 6078us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.400648 (+ 99us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2281442}
I20260505 07:54:01.401398 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 337) took 2299 ms. Trace:
I20260505 07:54:01.401525 1178 rpcz_store.cc:276] 0505 07:53:59.101750 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.102044 (+ 294us) service_pool.cc:224] Handling call
0505 07:54:01.401265 (+2299221us) ts_tablet_manager.cc:1151] Deleting tablet e29ba55a846548679cb3b86305d0d6ed
0505 07:54:01.401273 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.401367 (+ 94us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2265268}
I20260505 07:54:01.401696 1176 tablet_bootstrap.cc:492] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.401961 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 338) took 2237 ms. Trace:
I20260505 07:54:01.402068 1178 rpcz_store.cc:276] 0505 07:53:59.164237 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.164300 (+ 63us) service_pool.cc:224] Handling call
0505 07:54:01.401853 (+2237553us) ts_tablet_manager.cc:1151] Deleting tablet 7001482d90364dbdb8b9c5bfb98e1cd4
0505 07:54:01.401861 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.401933 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2237285}
I20260505 07:54:01.402478 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 339) took 2056 ms. Trace:
I20260505 07:54:01.402585 1178 rpcz_store.cc:276] 0505 07:53:59.346517 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.346945 (+ 428us) service_pool.cc:224] Handling call
0505 07:54:01.402362 (+2055417us) ts_tablet_manager.cc:1151] Deleting tablet d7e0c8ad716e443588d4ae3aa28ed4bf
0505 07:54:01.402370 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.402450 (+ 80us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2055223}
I20260505 07:54:01.403177 1176 tablet_bootstrap.cc:654] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.403795 1178 tablet_replica.cc:333] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.404124 1178 raft_consensus.cc:2243] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.404356 1178 raft_consensus.cc:2272] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.404947 1178 ts_tablet_manager.cc:1916] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.409090 1181 tablet_bootstrap.cc:654] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:01.414263 991 consensus_peers.cc:597] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e85b44337f62494c997c0008fc492aab. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:01.415036 1171 ts_tablet_manager.cc:1929] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.415271 1171 log.cc:1199] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/909306e1e5f34b7c8fbc0d86cafce531
I20260505 07:54:01.416112 1171 ts_tablet_manager.cc:1950] T 909306e1e5f34b7c8fbc0d86cafce531 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.416989 1171 tablet_replica.cc:333] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:01.417503 1171 raft_consensus.cc:2243] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.417716 1171 raft_consensus.cc:2272] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.417945 1178 ts_tablet_manager.cc:1929] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.418074 1178 log.cc:1199] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/c6dcbebddcdc4ad9a788c01134e1101c
I20260505 07:54:01.418308 1171 ts_tablet_manager.cc:1916] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.418395 1178 ts_tablet_manager.cc:1950] T c6dcbebddcdc4ad9a788c01134e1101c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.419178 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 340) took 2072 ms. Trace:
I20260505 07:54:01.419267 1178 rpcz_store.cc:276] 0505 07:53:59.347482 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.347686 (+ 204us) service_pool.cc:224] Handling call
0505 07:54:01.403733 (+2056047us) ts_tablet_manager.cc:1151] Deleting tablet c6dcbebddcdc4ad9a788c01134e1101c
0505 07:54:01.403744 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.414371 (+ 10627us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.417885 (+ 3514us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.419128 (+ 1243us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2054387}
I20260505 07:54:01.425132 1172 tablet_bootstrap.cc:492] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.425190 1569 raft_consensus.cc:493] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.425316 1172 ts_tablet_manager.cc:1403] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.095s user 0.000s sys 0.004s
I20260505 07:54:01.425418 1569 raft_consensus.cc:515] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.426031 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 349) took 2029 ms. Trace:
I20260505 07:54:01.426282 1569 leader_election.cc:290] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:01.426337 1172 raft_consensus.cc:359] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.426554 1172 raft_consensus.cc:385] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.426256 1178 rpcz_store.cc:276] 0505 07:53:59.395761 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.395814 (+ 53us) service_pool.cc:224] Handling call
0505 07:54:01.424615 (+2028801us) ts_tablet_manager.cc:1151] Deleting tablet 8b406c4c6ba741bc85eef54c6f576459
0505 07:54:01.424626 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.425149 (+ 523us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2028576}
I20260505 07:54:01.426656 1172 raft_consensus.cc:740] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.426910 1172 consensus_queue.cc:260] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.427151 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ef4f516e64a44fd8cd33674fb808067" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.427433 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ef4f516e64a44fd8cd33674fb808067" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:01.427672 1172 ts_tablet_manager.cc:1434] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
I20260505 07:54:01.427970 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 350) took 2031 ms. Trace:
W20260505 07:54:01.428037 856 leader_election.cc:343] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.428097 1172 tablet_bootstrap.cc:492] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:01.429744 854 leader_election.cc:343] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.429868 854 leader_election.cc:304] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.430121 1569 raft_consensus.cc:2749] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.431660 1178 rpcz_store.cc:276] 0505 07:53:59.396854 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.396901 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:01.427707 (+2030806us) ts_tablet_manager.cc:1151] Deleting tablet a7d8265c05b74b28959d0de41fad1fde
0505 07:54:01.427716 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.427920 (+ 204us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2030623}
I20260505 07:54:01.432873 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 345) took 2076 ms. Trace:
I20260505 07:54:01.433012 1178 rpcz_store.cc:276] 0505 07:53:59.355668 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.356702 (+ 1034us) service_pool.cc:224] Handling call
0505 07:54:01.432294 (+2075592us) ts_tablet_manager.cc:1151] Deleting tablet 02412333633a4cdebe95a144fa6c363c
0505 07:54:01.432303 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.432384 (+ 81us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2013007}
I20260505 07:54:01.433435 1171 ts_tablet_manager.cc:1929] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.433651 1171 log.cc:1199] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/7219f9896fd94187ae714635077110af
I20260505 07:54:01.433753 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 346) took 2045 ms. Trace:
I20260505 07:54:01.434437 1181 tablet_bootstrap.cc:492] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.434692 1181 ts_tablet_manager.cc:1403] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.002s
I20260505 07:54:01.435622 1181 raft_consensus.cc:359] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.435858 1181 raft_consensus.cc:385] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.435981 1181 raft_consensus.cc:740] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.436234 1181 consensus_queue.cc:260] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.438328 1181 ts_tablet_manager.cc:1434] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260505 07:54:01.438642 1181 tablet_bootstrap.cc:492] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.440976 1172 tablet_bootstrap.cc:654] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.441118 1178 rpcz_store.cc:276] 0505 07:53:59.388532 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.439357 (+ 50825us) service_pool.cc:224] Handling call
0505 07:54:01.433622 (+1994265us) ts_tablet_manager.cc:1151] Deleting tablet 067c32ecccb44bca8bf9cc3f6c229326
0505 07:54:01.433631 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.433707 (+ 76us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1994020}
I20260505 07:54:01.442569 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 352) took 2039 ms. Trace:
I20260505 07:54:01.442720 1178 rpcz_store.cc:276] 0505 07:53:59.403458 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.439627 (+ 36169us) service_pool.cc:224] Handling call
0505 07:54:01.442406 (+2002779us) ts_tablet_manager.cc:1151] Deleting tablet 3075e87e078f4d2aa1d329d18902354b
0505 07:54:01.442415 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.442518 (+ 103us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2002618}
I20260505 07:54:01.443097 1178 tablet_replica.cc:333] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.443315 1178 raft_consensus.cc:2243] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.443559 1178 raft_consensus.cc:2272] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.442276 1181 tablet_bootstrap.cc:654] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.445120 1178 ts_tablet_manager.cc:1916] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.445616 1171 ts_tablet_manager.cc:1950] T 7219f9896fd94187ae714635077110af P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.447762 1171 tablet_replica.cc:333] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:01.448081 1171 raft_consensus.cc:2243] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.448344 1171 raft_consensus.cc:2272] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.449087 1171 ts_tablet_manager.cc:1916] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.453699 1178 ts_tablet_manager.cc:1929] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.455941 1171 ts_tablet_manager.cc:1929] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.456115 1171 log.cc:1199] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/1838f89dc14a47f99f95f4cc5fdefd70
I20260505 07:54:01.456511 1171 ts_tablet_manager.cc:1950] T 1838f89dc14a47f99f95f4cc5fdefd70 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.458060 1171 tablet_replica.cc:333] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:01.458581 1171 raft_consensus.cc:2243] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.458863 1171 raft_consensus.cc:2272] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.459620 1171 ts_tablet_manager.cc:1916] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.461097 1172 tablet_bootstrap.cc:492] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.461320 1172 ts_tablet_manager.cc:1403] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.000s
I20260505 07:54:01.461550 1181 tablet_bootstrap.cc:492] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.461791 1181 ts_tablet_manager.cc:1403] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.005s
I20260505 07:54:01.462294 1172 raft_consensus.cc:359] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.462569 1172 raft_consensus.cc:385] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.462512 1181 raft_consensus.cc:359] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.462719 1181 raft_consensus.cc:385] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.462738 1172 raft_consensus.cc:740] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.462808 1181 raft_consensus.cc:740] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.463076 1181 consensus_queue.cc:260] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.463099 1172 consensus_queue.cc:260] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.465579 1172 ts_tablet_manager.cc:1434] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:01.466923 1176 tablet_bootstrap.cc:492] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.467144 1176 ts_tablet_manager.cc:1403] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.065s user 0.016s sys 0.000s
I20260505 07:54:01.467468 1181 ts_tablet_manager.cc:1434] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260505 07:54:01.468091 1176 raft_consensus.cc:359] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.468384 1176 raft_consensus.cc:385] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.468729 1176 raft_consensus.cc:740] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.469046 1176 consensus_queue.cc:260] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.469571 1176 ts_tablet_manager.cc:1434] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:01.470394 1181 tablet_bootstrap.cc:492] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.472079 1176 tablet_bootstrap.cc:492] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.472158 1181 tablet_bootstrap.cc:654] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.473047 1178 log.cc:1199] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/25a74b65d5d6459293ab4461f4d34cf2
I20260505 07:54:01.474447 1178 ts_tablet_manager.cc:1950] T 25a74b65d5d6459293ab4461f4d34cf2 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.475235 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 361) took 1967 ms. Trace:
I20260505 07:54:01.475404 1178 rpcz_store.cc:276] 0505 07:53:59.508068 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.508572 (+ 504us) service_pool.cc:224] Handling call
0505 07:54:01.443027 (+1934455us) ts_tablet_manager.cc:1151] Deleting tablet 25a74b65d5d6459293ab4461f4d34cf2
0505 07:54:01.443035 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.450493 (+ 7458us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.453621 (+ 3128us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.475200 (+ 21579us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1930348}
I20260505 07:54:01.477008 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 363) took 1915 ms. Trace:
I20260505 07:54:01.477180 1178 rpcz_store.cc:276] 0505 07:53:59.562016 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.562415 (+ 399us) service_pool.cc:224] Handling call
0505 07:54:01.476861 (+1914446us) ts_tablet_manager.cc:1151] Deleting tablet 4f3e076892234c1494b1a5c13cbfd2e2
0505 07:54:01.476870 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.476968 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1914185}
I20260505 07:54:01.477756 1512 raft_consensus.cc:493] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.477774 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 371) took 1828 ms. Trace:
I20260505 07:54:01.477885 1178 rpcz_store.cc:276] 0505 07:53:59.648865 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.648916 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:01.477604 (+1828688us) ts_tablet_manager.cc:1151] Deleting tablet 0459f2d45a2d44f1a73e0a0fff53866c
0505 07:54:01.477612 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.477694 (+ 82us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1828485}
I20260505 07:54:01.477931 1512 raft_consensus.cc:515] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:01.478520 1512 leader_election.cc:290] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:01.479841 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b404e50ae092419c9396772b3b14eeaa" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:01.483829 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 373) took 1810 ms. Trace:
I20260505 07:54:01.483908 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b404e50ae092419c9396772b3b14eeaa" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:01.483996 1178 rpcz_store.cc:276] 0505 07:53:59.673524 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.673585 (+ 61us) service_pool.cc:224] Handling call
0505 07:54:01.483694 (+1810109us) ts_tablet_manager.cc:1151] Deleting tablet f121fdd001ec4ea38ccd24a6afd9e51c
0505 07:54:01.483703 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.483798 (+ 95us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1809825}
I20260505 07:54:01.484407 1178 tablet_replica.cc:333] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.484653 1178 raft_consensus.cc:2243] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.484871 1178 raft_consensus.cc:2272] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.485775 1178 ts_tablet_manager.cc:1916] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.486160 1176 tablet_bootstrap.cc:654] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.490101 1181 tablet_bootstrap.cc:492] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.490280 1181 ts_tablet_manager.cc:1403] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.000s
I20260505 07:54:01.491493 1181 raft_consensus.cc:359] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.491689 921 raft_consensus.cc:2393] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:01.491748 1181 raft_consensus.cc:385] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.491856 1181 raft_consensus.cc:740] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.492062 1181 consensus_queue.cc:260] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.492472 718 leader_election.cc:304] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:01.493013 1512 raft_consensus.cc:3060] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.495889 1181 ts_tablet_manager.cc:1434] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:01.495919 1512 raft_consensus.cc:2749] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.496227 1181 tablet_bootstrap.cc:492] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.498029 1181 tablet_bootstrap.cc:654] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.498314 1596 consensus_queue.cc:1048] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:01.491667 1172 tablet_bootstrap.cc:492] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.501516 1172 tablet_bootstrap.cc:654] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.511979 1178 ts_tablet_manager.cc:1929] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.512526 1178 log.cc:1199] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/3d83e9d4243e409794ffe8a342bce6b3
I20260505 07:54:01.513141 1178 ts_tablet_manager.cc:1950] T 3d83e9d4243e409794ffe8a342bce6b3 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.514606 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 374) took 1840 ms. Trace:
I20260505 07:54:01.515614 1178 rpcz_store.cc:276] 0505 07:53:59.674122 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.674165 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:01.484351 (+1810186us) ts_tablet_manager.cc:1151] Deleting tablet 3d83e9d4243e409794ffe8a342bce6b3
0505 07:54:01.484360 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.499427 (+ 15067us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.511896 (+ 12469us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.514552 (+ 2656us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1809980}
I20260505 07:54:01.524431 1181 tablet_bootstrap.cc:492] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.527832 1181 ts_tablet_manager.cc:1403] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.032s user 0.005s sys 0.000s
I20260505 07:54:01.528788 1181 raft_consensus.cc:359] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.529063 1181 raft_consensus.cc:385] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.529178 1181 raft_consensus.cc:740] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.529548 1181 consensus_queue.cc:260] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.531972 1171 ts_tablet_manager.cc:1929] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.533766 1181 ts_tablet_manager.cc:1434] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:54:01.533980 1176 tablet_bootstrap.cc:492] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.534134 1176 ts_tablet_manager.cc:1403] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.062s user 0.002s sys 0.007s
I20260505 07:54:01.534929 1176 raft_consensus.cc:359] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.535115 1176 raft_consensus.cc:385] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.535521 1181 tablet_bootstrap.cc:492] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.537925 1181 tablet_bootstrap.cc:654] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.542348 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 375) took 1867 ms. Trace:
I20260505 07:54:01.542752 1580 consensus_queue.cc:1048] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:01.542526 1178 rpcz_store.cc:276] 0505 07:53:59.674686 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.674732 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:01.536266 (+1861534us) ts_tablet_manager.cc:1151] Deleting tablet 4783dd0b2fbe40219852d850fc0d9a82
0505 07:54:01.542179 (+ 5913us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.542289 (+ 110us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1861378}
I20260505 07:54:01.545022 1178 tablet_replica.cc:333] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.545282 1178 raft_consensus.cc:2243] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:01.545612 1178 pending_rounds.cc:70] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:54:01.545727 1178 pending_rounds.cc:77] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555454599499776 op_type: NO_OP noop_request { }
I20260505 07:54:01.545915 1178 raft_consensus.cc:2889] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:01.546051 1178 raft_consensus.cc:2272] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.548076 1178 ts_tablet_manager.cc:1916] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.550711 1171 log.cc:1199] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/3d6494590052459485059f5dce46f34f
I20260505 07:54:01.551272 1171 ts_tablet_manager.cc:1950] T 3d6494590052459485059f5dce46f34f P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.552736 1171 tablet_replica.cc:333] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:01.553037 1171 raft_consensus.cc:2243] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.554288 1178 ts_tablet_manager.cc:1929] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:01.560493 1178 log.cc:1199] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/e85b44337f62494c997c0008fc492aab
I20260505 07:54:01.561560 1172 tablet_bootstrap.cc:492] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.561754 1172 ts_tablet_manager.cc:1403] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.096s user 0.002s sys 0.002s
I20260505 07:54:01.561978 1176 raft_consensus.cc:740] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.562284 1176 consensus_queue.cc:260] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.562897 1176 ts_tablet_manager.cc:1434] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.029s user 0.000s sys 0.002s
I20260505 07:54:01.563192 1176 tablet_bootstrap.cc:492] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.565928 1172 raft_consensus.cc:359] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.566196 1172 raft_consensus.cc:385] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.567407 1172 raft_consensus.cc:740] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.567740 1178 ts_tablet_manager.cc:1950] T e85b44337f62494c997c0008fc492aab P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.567836 1176 tablet_bootstrap.cc:654] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.567759 1172 consensus_queue.cc:260] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.570107 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 364) took 1976 ms. Trace:
I20260505 07:54:01.567258 1181 tablet_bootstrap.cc:492] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.570398 1181 ts_tablet_manager.cc:1403] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.035s user 0.001s sys 0.005s
I20260505 07:54:01.571074 1172 ts_tablet_manager.cc:1434] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260505 07:54:01.572466 1181 raft_consensus.cc:359] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.572695 1181 raft_consensus.cc:385] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.572830 1181 raft_consensus.cc:740] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.573072 1181 consensus_queue.cc:260] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.573557 1181 ts_tablet_manager.cc:1434] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:01.574299 1172 tablet_bootstrap.cc:492] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.574397 1171 raft_consensus.cc:2272] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.575114 1181 tablet_bootstrap.cc:492] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.578454 1181 tablet_bootstrap.cc:654] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.578609 1178 rpcz_store.cc:276] 0505 07:53:59.593593 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.703668 (+110075us) service_pool.cc:224] Handling call
0505 07:54:01.544946 (+1841278us) ts_tablet_manager.cc:1151] Deleting tablet e85b44337f62494c997c0008fc492aab
0505 07:54:01.544955 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.551419 (+ 6464us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.554233 (+ 2814us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.570056 (+ 15823us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1841044}
I20260505 07:54:01.583455 1176 tablet_bootstrap.cc:492] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.581672 1596 raft_consensus.cc:493] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.583654 1176 ts_tablet_manager.cc:1403] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.001s
I20260505 07:54:01.583778 1596 raft_consensus.cc:515] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.584431 1596 leader_election.cc:290] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.584405 1176 raft_consensus.cc:359] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.584626 1176 raft_consensus.cc:385] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.580276 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 379) took 1841 ms. Trace:
I20260505 07:54:01.584718 1176 raft_consensus.cc:740] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.584892 1178 rpcz_store.cc:276] 0505 07:53:59.738858 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.738954 (+ 96us) service_pool.cc:224] Handling call
0505 07:54:01.580077 (+1841123us) ts_tablet_manager.cc:1151] Deleting tablet 472441bd29704be59aabb0f7a4964bb0
0505 07:54:01.580092 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.580207 (+ 115us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1840826}
I20260505 07:54:01.584982 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f88a5c0d0e2c436c8d8b6afc1f3c2edf" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.584959 1176 consensus_queue.cc:260] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.585191 781 raft_consensus.cc:2468] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:01.585546 1176 ts_tablet_manager.cc:1434] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:01.585562 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 385) took 1669 ms. Trace:
I20260505 07:54:01.585644 1178 rpcz_store.cc:276] 0505 07:53:59.915971 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.916032 (+ 61us) service_pool.cc:224] Handling call
0505 07:54:01.585395 (+1669363us) ts_tablet_manager.cc:1151] Deleting tablet 9044763bed1a41739ecce4def4cca1fe
0505 07:54:01.585403 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.585514 (+ 111us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1669059}
I20260505 07:54:01.585949 1176 tablet_bootstrap.cc:492] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.586127 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 387) took 1660 ms. Trace:
I20260505 07:54:01.586212 1178 rpcz_store.cc:276] 0505 07:53:59.925582 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.925637 (+ 55us) service_pool.cc:224] Handling call
0505 07:54:01.585987 (+1660350us) ts_tablet_manager.cc:1151] Deleting tablet a7fe9c7199cc4039ae877dc3d4aab17e
0505 07:54:01.585996 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.586077 (+ 81us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1660098}
I20260505 07:54:01.586565 1178 tablet_replica.cc:333] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.586742 1178 raft_consensus.cc:2243] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.586917 1178 raft_consensus.cc:2272] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.587430 1178 ts_tablet_manager.cc:1916] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.587960 1176 tablet_bootstrap.cc:654] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.589495 988 leader_election.cc:304] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:01.589972 1596 raft_consensus.cc:2804] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:01.590088 1596 raft_consensus.cc:493] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:01.590210 1596 raft_consensus.cc:3060] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.595332 1596 raft_consensus.cc:515] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.596827 1181 tablet_bootstrap.cc:492] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.597021 1181 ts_tablet_manager.cc:1403] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.001s
I20260505 07:54:01.597731 1181 raft_consensus.cc:359] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.597967 1181 raft_consensus.cc:385] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.597954 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f88a5c0d0e2c436c8d8b6afc1f3c2edf" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:01.598068 1181 raft_consensus.cc:740] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.598208 781 raft_consensus.cc:3060] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.598297 1181 consensus_queue.cc:260] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.598718 1181 ts_tablet_manager.cc:1434] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:01.599025 1181 tablet_bootstrap.cc:492] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.600669 1181 tablet_bootstrap.cc:654] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.604614 1171 ts_tablet_manager.cc:1916] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.605841 1172 tablet_bootstrap.cc:654] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.606642 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f88a5c0d0e2c436c8d8b6afc1f3c2edf" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:01.607942 919 raft_consensus.cc:2468] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:01.608947 1178 ts_tablet_manager.cc:1929] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.609176 781 raft_consensus.cc:2468] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:01.609781 988 leader_election.cc:304] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:01.610277 1580 raft_consensus.cc:2804] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:01.610414 1580 raft_consensus.cc:697] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:01.610670 1580 consensus_queue.cc:237] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.611970 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f88a5c0d0e2c436c8d8b6afc1f3c2edf" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:01.612191 919 raft_consensus.cc:3060] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.616530 919 raft_consensus.cc:2468] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:01.617228 1596 leader_election.cc:290] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.626754 1178 log.cc:1199] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/faf1140fe05e4f9493563ab8464c582b
I20260505 07:54:01.627131 1178 ts_tablet_manager.cc:1950] T faf1140fe05e4f9493563ab8464c582b P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.627919 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 389) took 1685 ms. Trace:
I20260505 07:54:01.627938 1171 ts_tablet_manager.cc:1929] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.628062 1171 log.cc:1199] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/7e707143a4e64725aa66db05b24999a6
I20260505 07:54:01.628044 1178 rpcz_store.cc:276] 0505 07:53:59.942790 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.942874 (+ 84us) service_pool.cc:224] Handling call
0505 07:54:01.586491 (+1643617us) ts_tablet_manager.cc:1151] Deleting tablet faf1140fe05e4f9493563ab8464c582b
0505 07:54:01.586499 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.590704 (+ 4205us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.608866 (+ 18162us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.627873 (+ 19007us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1643382}
I20260505 07:54:01.628448 1178 tablet_replica.cc:333] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.628680 1178 raft_consensus.cc:2243] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.628708 1171 ts_tablet_manager.cc:1950] T 7e707143a4e64725aa66db05b24999a6 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.629371 1178 raft_consensus.cc:2272] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.629930 1178 ts_tablet_manager.cc:1916] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.632925 1181 tablet_bootstrap.cc:492] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.633064 1181 ts_tablet_manager.cc:1403] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.034s user 0.001s sys 0.005s
I20260505 07:54:01.633634 1171 tablet_replica.cc:333] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:01.633893 1171 raft_consensus.cc:2243] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.633900 1181 raft_consensus.cc:359] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.634068 1181 raft_consensus.cc:385] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.634158 1181 raft_consensus.cc:740] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.634213 1171 raft_consensus.cc:2272] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.634372 1181 consensus_queue.cc:260] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.635037 1171 ts_tablet_manager.cc:1916] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.637832 1181 ts_tablet_manager.cc:1434] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:54:01.638708 1172 tablet_bootstrap.cc:492] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.638885 1172 ts_tablet_manager.cc:1403] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.068s user 0.001s sys 0.003s
I20260505 07:54:01.639731 1172 raft_consensus.cc:359] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.639930 1172 raft_consensus.cc:385] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.640000 1172 raft_consensus.cc:740] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.640237 1172 consensus_queue.cc:260] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.640770 1172 ts_tablet_manager.cc:1434] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:01.641129 1172 tablet_bootstrap.cc:492] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.642966 1512 raft_consensus.cc:493] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.643134 1512 raft_consensus.cc:515] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.643903 1512 leader_election.cc:290] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.644260 1172 tablet_bootstrap.cc:654] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.645545 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9959be9f83c2444282d012b61eef49b5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:01.645773 1055 raft_consensus.cc:2468] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:01.645967 1181 tablet_bootstrap.cc:492] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.647640 717 leader_election.cc:304] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
W20260505 07:54:01.647802 718 leader_election.cc:343] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 9959be9f83c2444282d012b61eef49b5
I20260505 07:54:01.650668 1512 raft_consensus.cc:2804] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:01.650800 1512 raft_consensus.cc:493] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:01.650893 1512 raft_consensus.cc:3060] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.653477 1512 raft_consensus.cc:515] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.654364 1178 ts_tablet_manager.cc:1929] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.654484 1178 log.cc:1199] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/3ac06bb9d273457ead843f5dd754539c
I20260505 07:54:01.654834 1178 ts_tablet_manager.cc:1950] T 3ac06bb9d273457ead843f5dd754539c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.655280 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9959be9f83c2444282d012b61eef49b5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:01.655449 1055 raft_consensus.cc:3060] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.658246 1512 leader_election.cc:290] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:01.659211 718 leader_election.cc:343] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 9959be9f83c2444282d012b61eef49b5
I20260505 07:54:01.666122 1512 raft_consensus.cc:493] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.666283 1512 raft_consensus.cc:515] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.666915 1512 leader_election.cc:290] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.656029 1181 tablet_bootstrap.cc:654] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.667770 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03f31efbc98c484c950c70d5e84d2136" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:01.667845 718 leader_election.cc:343] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 03f31efbc98c484c950c70d5e84d2136
I20260505 07:54:01.667982 1052 raft_consensus.cc:2468] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:01.657984 1055 raft_consensus.cc:2468] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:01.668510 1172 tablet_bootstrap.cc:492] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.668664 1172 ts_tablet_manager.cc:1403] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.002s
I20260505 07:54:01.668681 717 leader_election.cc:304] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:01.669068 717 leader_election.cc:304] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:01.669483 1512 raft_consensus.cc:2804] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:01.669592 1512 raft_consensus.cc:493] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:01.669672 1512 raft_consensus.cc:3060] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.669688 1172 raft_consensus.cc:359] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.670001 1172 raft_consensus.cc:385] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.670118 1172 raft_consensus.cc:740] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.670380 1172 consensus_queue.cc:260] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.655563 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 390) took 1712 ms. Trace:
I20260505 07:54:01.670949 1178 rpcz_store.cc:276] 0505 07:53:59.942924 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.943144 (+ 220us) service_pool.cc:224] Handling call
0505 07:54:01.628351 (+1685207us) ts_tablet_manager.cc:1151] Deleting tablet 3ac06bb9d273457ead843f5dd754539c
0505 07:54:01.628362 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.651334 (+ 22972us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.654306 (+ 2972us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.655514 (+ 1208us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1684979}
I20260505 07:54:01.671870 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 391) took 1726 ms. Trace:
I20260505 07:54:01.671953 1178 rpcz_store.cc:276] 0505 07:53:59.945252 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.945360 (+ 108us) service_pool.cc:224] Handling call
0505 07:54:01.671619 (+1726259us) ts_tablet_manager.cc:1151] Deleting tablet 36b33d265ba04e93993c7e0a4efea1a4
0505 07:54:01.671629 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.671824 (+ 195us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1726011}
I20260505 07:54:01.672755 1172 ts_tablet_manager.cc:1434] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:01.673044 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 392) took 1722 ms. Trace:
I20260505 07:54:01.673098 1172 tablet_bootstrap.cc:492] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.673125 1178 rpcz_store.cc:276] 0505 07:53:59.950849 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.950945 (+ 96us) service_pool.cc:224] Handling call
0505 07:54:01.672873 (+1721928us) ts_tablet_manager.cc:1151] Deleting tablet f1058fcb9fa6457a85489049ca1d9ada
0505 07:54:01.672881 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.672996 (+ 115us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1721698}
I20260505 07:54:01.674050 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 393) took 1723 ms. Trace:
I20260505 07:54:01.674412 1178 rpcz_store.cc:276] 0505 07:53:59.951044 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.951212 (+ 168us) service_pool.cc:224] Handling call
0505 07:54:01.673927 (+1722715us) ts_tablet_manager.cc:1151] Deleting tablet 0db162dfad2448e8ac428b3d3fbe17f7
0505 07:54:01.673934 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.674005 (+ 71us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1722562}
I20260505 07:54:01.675154 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 394) took 1722 ms. Trace:
I20260505 07:54:01.675524 1178 rpcz_store.cc:276] 0505 07:53:59.952507 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.952564 (+ 57us) service_pool.cc:224] Handling call
0505 07:54:01.675032 (+1722468us) ts_tablet_manager.cc:1151] Deleting tablet 297537a92684425d8508f7b3b4c9d379
0505 07:54:01.675041 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.675109 (+ 68us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1722211}
I20260505 07:54:01.676216 1178 tablet_replica.cc:333] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.676622 1178 raft_consensus.cc:2243] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.677112 1178 raft_consensus.cc:2272] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.677970 1178 ts_tablet_manager.cc:1916] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.678675 1512 raft_consensus.cc:515] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.679064 1172 tablet_bootstrap.cc:654] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.679847 1512 leader_election.cc:290] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.679518 1171 ts_tablet_manager.cc:1929] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.680089 1512 raft_consensus.cc:2804] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:01.680109 1171 log.cc:1199] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/1ef4f516e64a44fd8cd33674fb808067
I20260505 07:54:01.680207 1512 raft_consensus.cc:697] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:01.680662 1171 ts_tablet_manager.cc:1950] T 1ef4f516e64a44fd8cd33674fb808067 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.680493 1512 consensus_queue.cc:237] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.682722 1171 tablet_replica.cc:333] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:01.682945 1171 raft_consensus.cc:2243] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.687851 1171 raft_consensus.cc:2272] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.688601 1171 ts_tablet_manager.cc:1916] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.688851 1580 raft_consensus.cc:493] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.689010 1580 raft_consensus.cc:515] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.689601 1580 leader_election.cc:290] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.689783 1645 raft_consensus.cc:493] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.689968 1645 raft_consensus.cc:515] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.690342 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03f31efbc98c484c950c70d5e84d2136" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:01.690550 1055 raft_consensus.cc:3060] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.690639 1645 leader_election.cc:290] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:01.690645 991 leader_election.cc:343] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: ccdde3e5a13e436d9f2328831e155adb
I20260505 07:54:01.690722 1569 raft_consensus.cc:493] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.690855 1569 raft_consensus.cc:515] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.691468 1569 leader_election.cc:290] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:01.691936 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf76e868bbcb44b0b1887c84dd27f093" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.692008 1181 tablet_bootstrap.cc:492] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
W20260505 07:54:01.692831 854 leader_election.cc:343] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.693186 1176 tablet_bootstrap.cc:492] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.693348 1176 ts_tablet_manager.cc:1403] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.107s user 0.014s sys 0.037s
I20260505 07:54:01.694398 1176 raft_consensus.cc:359] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.694648 1176 raft_consensus.cc:385] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.694766 1176 raft_consensus.cc:740] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.696710 1181 ts_tablet_manager.cc:1403] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.046s user 0.003s sys 0.012s
I20260505 07:54:01.697328 1176 consensus_queue.cc:260] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.697642 1181 raft_consensus.cc:359] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.697822 1176 ts_tablet_manager.cc:1434] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:01.698309 1181 raft_consensus.cc:385] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.698392 1181 raft_consensus.cc:740] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.699036 1580 raft_consensus.cc:493] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.699002 1181 consensus_queue.cc:260] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.699198 1580 raft_consensus.cc:515] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.699566 1181 ts_tablet_manager.cc:1434] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:01.700453 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccdde3e5a13e436d9f2328831e155adb" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.700641 781 raft_consensus.cc:2468] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:01.700824 1181 tablet_bootstrap.cc:492] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:54:01.701026 718 leader_election.cc:343] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 03f31efbc98c484c950c70d5e84d2136
I20260505 07:54:01.701035 1055 raft_consensus.cc:2468] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:01.701221 718 leader_election.cc:343] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 3d83e9d4243e409794ffe8a342bce6b3
I20260505 07:54:01.701645 988 leader_election.cc:304] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:01.701942 1596 raft_consensus.cc:2804] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260505 07:54:01.702592 717 leader_election.cc:343] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 3d83e9d4243e409794ffe8a342bce6b3
I20260505 07:54:01.702749 717 leader_election.cc:304] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:01.703220 1596 raft_consensus.cc:493] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:01.702055 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf76e868bbcb44b0b1887c84dd27f093" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:01.703332 1596 raft_consensus.cc:3060] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.704114 1176 tablet_bootstrap.cc:492] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.706317 1176 tablet_bootstrap.cc:654] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.706459 1596 raft_consensus.cc:515] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:01.706627 856 leader_election.cc:343] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.706750 856 leader_election.cc:304] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.707114 1569 raft_consensus.cc:2749] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.707238 1646 raft_consensus.cc:2749] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:01.707832 991 leader_election.cc:343] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: f92c53c68a4b4e319e98b84128e8e6a4
W20260505 07:54:01.708547 991 leader_election.cc:343] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: ccdde3e5a13e436d9f2328831e155adb
I20260505 07:54:01.707114 1596 leader_election.cc:290] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.709227 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92c53c68a4b4e319e98b84128e8e6a4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.710036 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccdde3e5a13e436d9f2328831e155adb" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:01.710187 781 raft_consensus.cc:3060] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.709242 1580 leader_election.cc:290] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.712643 781 raft_consensus.cc:2468] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:01.709573 1178 ts_tablet_manager.cc:1929] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.712970 1178 log.cc:1199] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/6429919aba17447dbdb393176360bbb0
I20260505 07:54:01.713428 1178 ts_tablet_manager.cc:1950] T 6429919aba17447dbdb393176360bbb0 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.713840 1580 raft_consensus.cc:493] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.713976 1580 raft_consensus.cc:515] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.714255 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 388) took 1771 ms. Trace:
I20260505 07:54:01.712414 1181 tablet_bootstrap.cc:654] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.714819 1580 leader_election.cc:290] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:01.716043 991 leader_election.cc:343] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 32c008bd99224a7595cecc607e5496a9
I20260505 07:54:01.714839 1178 rpcz_store.cc:276] 0505 07:53:59.942589 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:53:59.960692 (+ 18103us) service_pool.cc:224] Handling call
0505 07:54:01.676147 (+1715455us) ts_tablet_manager.cc:1151] Deleting tablet 6429919aba17447dbdb393176360bbb0
0505 07:54:01.676155 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.682088 (+ 5933us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.709047 (+ 26959us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.714214 (+ 5167us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1715173}
I20260505 07:54:01.716874 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 409) took 1419 ms. Trace:
I20260505 07:54:01.717224 1178 rpcz_store.cc:276] 0505 07:54:00.297259 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.297314 (+ 55us) service_pool.cc:224] Handling call
0505 07:54:01.716714 (+1419400us) ts_tablet_manager.cc:1151] Deleting tablet e82e9ae417784b7e83f4843b129d0580
0505 07:54:01.716727 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.716839 (+ 112us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1417026}
I20260505 07:54:01.717816 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 415) took 1361 ms. Trace:
I20260505 07:54:01.717955 1178 rpcz_store.cc:276] 0505 07:54:00.356102 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.356947 (+ 845us) service_pool.cc:224] Handling call
0505 07:54:01.717697 (+1360750us) ts_tablet_manager.cc:1151] Deleting tablet 6ca2852cee84476dbe5e5072a43087a1
0505 07:54:01.717707 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.717777 (+ 70us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1360426}
I20260505 07:54:01.718420 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 418) took 1358 ms. Trace:
I20260505 07:54:01.718554 1178 rpcz_store.cc:276] 0505 07:54:00.360096 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.360149 (+ 53us) service_pool.cc:224] Handling call
0505 07:54:01.718308 (+1358159us) ts_tablet_manager.cc:1151] Deleting tablet df1b62a30b2842b897a04c793bb5e7ce
0505 07:54:01.718316 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.718385 (+ 69us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1357981}
W20260505 07:54:01.718977 988 leader_election.cc:343] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.719058 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 417) took 1359 ms. Trace:
I20260505 07:54:01.719137 988 leader_election.cc:304] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.719241 1171 ts_tablet_manager.cc:1929] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.719342 1171 log.cc:1199] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/bd581a6799a14a8dafb80d411e8936f3
I20260505 07:54:01.721061 1172 tablet_bootstrap.cc:492] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.721299 1172 ts_tablet_manager.cc:1403] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.048s user 0.004s sys 0.004s
I20260505 07:54:01.722236 1172 raft_consensus.cc:359] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.722524 1172 raft_consensus.cc:385] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.722666 1172 raft_consensus.cc:740] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.722985 1172 consensus_queue.cc:260] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.723881 1172 ts_tablet_manager.cc:1434] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:01.719180 1178 rpcz_store.cc:276] 0505 07:54:00.359939 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.359991 (+ 52us) service_pool.cc:224] Handling call
0505 07:54:01.718883 (+1358892us) ts_tablet_manager.cc:1151] Deleting tablet b5a0cbf9c3d74cea92920189da332c1c
0505 07:54:01.718891 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.719024 (+ 133us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1358210}
I20260505 07:54:01.724320 1172 tablet_bootstrap.cc:492] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.724404 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 419) took 1363 ms. Trace:
I20260505 07:54:01.724502 1178 rpcz_store.cc:276] 0505 07:54:00.361350 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.361405 (+ 55us) service_pool.cc:224] Handling call
0505 07:54:01.724268 (+1362863us) ts_tablet_manager.cc:1151] Deleting tablet d77ff90ec2ff4b6eb1795d149459c23a
0505 07:54:01.724276 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.724373 (+ 97us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1362615}
I20260505 07:54:01.719664 1580 raft_consensus.cc:2749] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.720069 988 leader_election.cc:304] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:01.725107 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 416) took 1365 ms. Trace:
I20260505 07:54:01.726608 1178 rpcz_store.cc:276] 0505 07:54:00.359811 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.364035 (+ 4224us) service_pool.cc:224] Handling call
0505 07:54:01.724996 (+1360961us) ts_tablet_manager.cc:1151] Deleting tablet 02e06d8815034d91986cef12964860ea
0505 07:54:01.725004 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.725078 (+ 74us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1360389}
W20260505 07:54:01.726835 988 leader_election.cc:343] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 32c008bd99224a7595cecc607e5496a9
I20260505 07:54:01.726946 988 leader_election.cc:304] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.727026 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 422) took 1350 ms. Trace:
I20260505 07:54:01.727121 1178 rpcz_store.cc:276] 0505 07:54:00.376451 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.429725 (+ 53274us) service_pool.cc:224] Handling call
0505 07:54:01.726914 (+1297189us) ts_tablet_manager.cc:1151] Deleting tablet e010f23c4df647ab863e03c3cfed7898
0505 07:54:01.726922 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.726994 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1296836}
I20260505 07:54:01.727291 1567 raft_consensus.cc:2804] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:01.727566 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 420) took 1351 ms. Trace:
I20260505 07:54:01.727725 1178 rpcz_store.cc:276] 0505 07:54:00.376246 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.453809 (+ 77563us) service_pool.cc:224] Handling call
0505 07:54:01.727418 (+1273609us) ts_tablet_manager.cc:1151] Deleting tablet abe5f6f0774545b296bf1b03a08e145d
0505 07:54:01.727426 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.727541 (+ 115us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1273322}
I20260505 07:54:01.728135 1178 tablet_replica.cc:333] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.728372 1178 raft_consensus.cc:2243] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.728559 1178 raft_consensus.cc:2272] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.729090 1178 ts_tablet_manager.cc:1916] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.730404 1580 raft_consensus.cc:2749] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.724014 1596 raft_consensus.cc:493] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.730732 1596 raft_consensus.cc:515] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.731374 1596 leader_election.cc:290] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.731977 1596 raft_consensus.cc:493] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.732214 1176 tablet_bootstrap.cc:492] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.732138 1596 raft_consensus.cc:515] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:01.732276 991 leader_election.cc:343] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8a60c015136246a3b02f70254c0bb67f
I20260505 07:54:01.732378 1176 ts_tablet_manager.cc:1403] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.002s
I20260505 07:54:01.732818 1596 leader_election.cc:290] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.733251 1176 raft_consensus.cc:359] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.733434 1176 raft_consensus.cc:385] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260505 07:54:01.733435 991 leader_election.cc:343] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 7585f00119e14006990ed551667b4486
I20260505 07:54:01.733502 1176 raft_consensus.cc:740] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.733774 1176 consensus_queue.cc:260] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.734288 1176 ts_tablet_manager.cc:1434] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:01.734638 1176 tablet_bootstrap.cc:492] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.736183 1176 tablet_bootstrap.cc:654] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.737886 1172 tablet_bootstrap.cc:654] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.738385 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a60c015136246a3b02f70254c0bb67f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.738611 781 raft_consensus.cc:2468] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:01.738744 1171 ts_tablet_manager.cc:1950] T bd581a6799a14a8dafb80d411e8936f3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.739490 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585f00119e14006990ed551667b4486" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.739969 988 leader_election.cc:304] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:01.740353 1596 raft_consensus.cc:2804] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:01.740447 1596 raft_consensus.cc:493] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:01.740569 1596 raft_consensus.cc:3060] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.743168 1596 raft_consensus.cc:515] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.744009 1596 leader_election.cc:290] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.744256 1171 tablet_replica.cc:333] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:01.744482 1171 raft_consensus.cc:2243] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.744659 1171 raft_consensus.cc:2272] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:01.744776 991 leader_election.cc:343] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8a60c015136246a3b02f70254c0bb67f
W20260505 07:54:01.744946 988 leader_election.cc:343] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.745087 988 leader_election.cc:304] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.745419 1596 raft_consensus.cc:2749] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.746466 1176 tablet_bootstrap.cc:492] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.746644 1176 ts_tablet_manager.cc:1403] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260505 07:54:01.746613 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a60c015136246a3b02f70254c0bb67f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:01.746869 781 raft_consensus.cc:3060] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.747540 1176 raft_consensus.cc:359] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.748054 1176 raft_consensus.cc:385] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.748135 1176 raft_consensus.cc:740] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.748337 1176 consensus_queue.cc:260] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.748826 1176 ts_tablet_manager.cc:1434] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:01.750193 781 raft_consensus.cc:2468] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:01.750626 988 leader_election.cc:304] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:01.751060 1596 raft_consensus.cc:2804] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:01.751318 1596 raft_consensus.cc:697] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:01.751854 1596 consensus_queue.cc:237] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.752170 1567 raft_consensus.cc:697] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:01.752439 1171 ts_tablet_manager.cc:1916] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.752455 1567 consensus_queue.cc:237] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.756232 1178 ts_tablet_manager.cc:1929] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.756633 1176 tablet_bootstrap.cc:492] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.757524 1172 tablet_bootstrap.cc:492] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.757725 1172 ts_tablet_manager.cc:1403] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.000s
I20260505 07:54:01.758350 1176 tablet_bootstrap.cc:654] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.758597 1172 raft_consensus.cc:359] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.758870 1172 raft_consensus.cc:385] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.759008 1172 raft_consensus.cc:740] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.756372 1178 log.cc:1199] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/9862b87d74974dc49d7dc9049b2a694a
I20260505 07:54:01.759403 1172 consensus_queue.cc:260] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.759593 1178 ts_tablet_manager.cc:1950] T 9862b87d74974dc49d7dc9049b2a694a P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.760187 1172 ts_tablet_manager.cc:1434] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:01.760514 1172 tablet_bootstrap.cc:492] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.761608 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 426) took 1298 ms. Trace:
I20260505 07:54:01.761698 1178 rpcz_store.cc:276] 0505 07:54:00.462959 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.464724 (+ 1765us) service_pool.cc:224] Handling call
0505 07:54:01.728083 (+1263359us) ts_tablet_manager.cc:1151] Deleting tablet 9862b87d74974dc49d7dc9049b2a694a
0505 07:54:01.728091 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.738895 (+ 10804us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.756161 (+ 17266us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.761559 (+ 5398us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1246205}
I20260505 07:54:01.765045 1172 tablet_bootstrap.cc:654] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.765108 1178 tablet_replica.cc:333] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.772249 1178 raft_consensus.cc:2243] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.772418 1569 raft_consensus.cc:493] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.772634 1178 raft_consensus.cc:2272] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.772574 1569 raft_consensus.cc:515] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.773141 1569 leader_election.cc:290] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:01.773545 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323269e708964848a94c64e6a781a860" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.774165 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323269e708964848a94c64e6a781a860" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:01.774201 854 leader_election.cc:343] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.774431 1178 ts_tablet_manager.cc:1916] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:01.774672 856 leader_election.cc:343] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.774482 1181 tablet_bootstrap.cc:492] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.774798 856 leader_election.cc:304] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.774863 1181 ts_tablet_manager.cc:1403] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.075s user 0.003s sys 0.008s
I20260505 07:54:01.775310 1569 raft_consensus.cc:2749] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.775684 1181 raft_consensus.cc:359] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.775872 1181 raft_consensus.cc:385] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.775967 1181 raft_consensus.cc:740] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.776234 1181 consensus_queue.cc:260] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.777035 1181 ts_tablet_manager.cc:1434] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260505 07:54:01.777307 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 74713.7us
I20260505 07:54:01.778626 1171 ts_tablet_manager.cc:1929] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.780457 717 leader_election.cc:304] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:01.783522 1178 ts_tablet_manager.cc:1929] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.783707 1171 log.cc:1199] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/bf76e868bbcb44b0b1887c84dd27f093
I20260505 07:54:01.784188 1171 ts_tablet_manager.cc:1950] T bf76e868bbcb44b0b1887c84dd27f093 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:01.784229 1649 raft_consensus.cc:2804] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:01.784789 1649 raft_consensus.cc:697] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:01.785213 1649 consensus_queue.cc:237] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.788407 1172 tablet_bootstrap.cc:492] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.788697 1172 ts_tablet_manager.cc:1403] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.002s
I20260505 07:54:01.789713 1172 raft_consensus.cc:359] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.789985 1172 raft_consensus.cc:385] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.790067 1172 raft_consensus.cc:740] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.790352 1172 consensus_queue.cc:260] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.791090 1172 ts_tablet_manager.cc:1434] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:01.792135 1172 tablet_bootstrap.cc:492] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.794170 1569 raft_consensus.cc:493] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.794312 1569 raft_consensus.cc:515] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.794879 1569 leader_election.cc:290] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:01.797175 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1058fcb9fa6457a85489049ca1d9ada" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:01.797770 854 leader_election.cc:343] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.801424 1178 log.cc:1199] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/8b0cf443e3044e56a26b0a8a61c9cf7d
I20260505 07:54:01.789085 1181 tablet_bootstrap.cc:492] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.795349 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1058fcb9fa6457a85489049ca1d9ada" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:01.801872 1178 ts_tablet_manager.cc:1950] T 8b0cf443e3044e56a26b0a8a61c9cf7d P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
W20260505 07:54:01.802043 856 leader_election.cc:343] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:01.802178 856 leader_election.cc:304] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:01.802603 1172 tablet_bootstrap.cc:654] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.802620 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 433) took 1315 ms. Trace:
I20260505 07:54:01.802613 1569 raft_consensus.cc:2749] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.802721 1178 rpcz_store.cc:276] 0505 07:54:00.486632 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.486869 (+ 237us) service_pool.cc:224] Handling call
0505 07:54:01.762065 (+1275196us) ts_tablet_manager.cc:1151] Deleting tablet 8b0cf443e3044e56a26b0a8a61c9cf7d
0505 07:54:01.762073 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.780500 (+ 18427us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.783433 (+ 2933us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.802585 (+ 19152us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1274970}
I20260505 07:54:01.803087 1178 tablet_replica.cc:333] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.803313 1178 raft_consensus.cc:2243] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.803486 1178 raft_consensus.cc:2272] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.804826 1176 tablet_bootstrap.cc:492] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.804857 1181 tablet_bootstrap.cc:654] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.804979 1176 ts_tablet_manager.cc:1403] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.048s user 0.003s sys 0.005s
I20260505 07:54:01.806226 1176 raft_consensus.cc:359] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.806417 1176 raft_consensus.cc:385] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.806483 1176 raft_consensus.cc:740] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.806710 1176 consensus_queue.cc:260] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.807160 1176 ts_tablet_manager.cc:1434] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:01.807475 1176 tablet_bootstrap.cc:492] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.811249 1181 tablet_bootstrap.cc:492] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.811828 1181 ts_tablet_manager.cc:1403] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.002s
I20260505 07:54:01.812943 1181 raft_consensus.cc:359] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:01.813659 1181 raft_consensus.cc:385] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.813773 1181 raft_consensus.cc:740] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.814049 1181 consensus_queue.cc:260] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:01.814153 1176 tablet_bootstrap.cc:654] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.814985 1181 ts_tablet_manager.cc:1434] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:01.821095 1181 tablet_bootstrap.cc:492] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.821543 1596 raft_consensus.cc:493] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.821678 1596 raft_consensus.cc:515] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.822221 1596 leader_election.cc:290] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.823673 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1852f7f4ff594639908b03730b68fe86" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:01.823905 781 raft_consensus.cc:2468] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:01.824318 1181 tablet_bootstrap.cc:654] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.824747 988 leader_election.cc:304] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:01.825112 1178 ts_tablet_manager.cc:1916] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:01.825295 991 leader_election.cc:343] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 1852f7f4ff594639908b03730b68fe86
I20260505 07:54:01.826023 1596 raft_consensus.cc:2804] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:01.826117 1596 raft_consensus.cc:493] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:01.826218 1596 raft_consensus.cc:3060] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.826579 1172 tablet_bootstrap.cc:492] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.826712 1172 ts_tablet_manager.cc:1403] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.000s
I20260505 07:54:01.828003 1172 raft_consensus.cc:359] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.828181 1172 raft_consensus.cc:385] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.828255 1172 raft_consensus.cc:740] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.828471 1172 consensus_queue.cc:260] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.829078 1172 ts_tablet_manager.cc:1434] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260505 07:54:01.829381 1172 tablet_bootstrap.cc:492] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.829461 1596 raft_consensus.cc:515] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.830654 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1852f7f4ff594639908b03730b68fe86" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:01.830845 781 raft_consensus.cc:3060] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.831148 1172 tablet_bootstrap.cc:654] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.835698 781 raft_consensus.cc:2468] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:01.836159 988 leader_election.cc:304] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:01.836468 1567 raft_consensus.cc:2804] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:01.837028 1567 raft_consensus.cc:697] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:01.837289 1567 consensus_queue.cc:237] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:01.840833 991 leader_election.cc:343] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 1852f7f4ff594639908b03730b68fe86
I20260505 07:54:01.841193 1596 leader_election.cc:290] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.844424 1176 tablet_bootstrap.cc:492] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.844578 1176 ts_tablet_manager.cc:1403] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.037s user 0.000s sys 0.003s
I20260505 07:54:01.845323 1176 raft_consensus.cc:359] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.845499 1176 raft_consensus.cc:385] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.845575 1176 raft_consensus.cc:740] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.849376 1176 consensus_queue.cc:260] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.849857 1176 ts_tablet_manager.cc:1434] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260505 07:54:01.858443 1176 tablet_bootstrap.cc:492] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.862111 1178 ts_tablet_manager.cc:1929] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:01.862442 1178 log.cc:1199] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/3272555a8c814c0ebe61ca32d715212b
I20260505 07:54:01.862838 1178 ts_tablet_manager.cc:1950] T 3272555a8c814c0ebe61ca32d715212b P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:01.863484 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 434) took 1376 ms. Trace:
I20260505 07:54:01.866580 1176 tablet_bootstrap.cc:654] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.867902 1172 tablet_bootstrap.cc:492] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.868150 1172 ts_tablet_manager.cc:1403] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.039s user 0.014s sys 0.004s
I20260505 07:54:01.868963 1172 raft_consensus.cc:359] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.869220 1172 raft_consensus.cc:385] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.869426 1172 raft_consensus.cc:740] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.869704 1172 consensus_queue.cc:260] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.870682 1172 ts_tablet_manager.cc:1434] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:01.871179 1172 tablet_bootstrap.cc:492] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.872911 1172 tablet_bootstrap.cc:654] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.876688 1176 tablet_bootstrap.cc:492] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.876863 1176 ts_tablet_manager.cc:1403] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.003s
I20260505 07:54:01.877629 1176 raft_consensus.cc:359] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.877854 1176 raft_consensus.cc:385] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.877947 1176 raft_consensus.cc:740] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.878221 1176 consensus_queue.cc:260] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.878786 1176 ts_tablet_manager.cc:1434] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:01.882408 1181 tablet_bootstrap.cc:492] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.882616 1181 ts_tablet_manager.cc:1403] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.067s user 0.007s sys 0.000s
I20260505 07:54:01.883926 1176 tablet_bootstrap.cc:492] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.885954 1176 tablet_bootstrap.cc:654] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.890486 1668 raft_consensus.cc:493] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28d3eda04f9c47ae976d19c218b6dcc5)
I20260505 07:54:01.890640 1668 raft_consensus.cc:515] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:54:01.892153 1181 raft_consensus.cc:359] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.892396 1181 raft_consensus.cc:385] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.893778 1181 raft_consensus.cc:740] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.893872 1176 tablet_bootstrap.cc:492] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.894006 1176 ts_tablet_manager.cc:1403] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260505 07:54:01.894039 1668 leader_election.cc:290] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565), 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401)
I20260505 07:54:01.894063 1181 consensus_queue.cc:260] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.894768 1181 ts_tablet_manager.cc:1434] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20260505 07:54:01.894757 1176 raft_consensus.cc:359] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.894965 1176 raft_consensus.cc:385] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.895059 1176 raft_consensus.cc:740] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.895255 1176 consensus_queue.cc:260] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.895788 1176 ts_tablet_manager.cc:1434] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:01.896083 1176 tablet_bootstrap.cc:492] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.897699 1176 tablet_bootstrap.cc:654] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.895165 1181 tablet_bootstrap.cc:492] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.899677 1172 tablet_bootstrap.cc:492] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.899856 1172 ts_tablet_manager.cc:1403] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.029s user 0.010s sys 0.012s
I20260505 07:54:01.900642 1172 raft_consensus.cc:359] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.900898 1172 raft_consensus.cc:385] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.901057 1172 raft_consensus.cc:740] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.901376 1172 consensus_queue.cc:260] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.903374 1172 ts_tablet_manager.cc:1434] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:01.903895 1181 tablet_bootstrap.cc:654] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.863595 1178 rpcz_store.cc:276] 0505 07:54:00.486739 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.487124 (+ 385us) service_pool.cc:224] Handling call
0505 07:54:01.803025 (+1315901us) ts_tablet_manager.cc:1151] Deleting tablet 3272555a8c814c0ebe61ca32d715212b
0505 07:54:01.803035 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.859807 (+ 56772us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.862041 (+ 2234us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:01.863431 (+ 1390us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1311472}
I20260505 07:54:01.906652 1172 tablet_bootstrap.cc:492] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.907037 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 440) took 1345 ms. Trace:
I20260505 07:54:01.907140 1178 rpcz_store.cc:276] 0505 07:54:00.561858 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.561946 (+ 88us) service_pool.cc:224] Handling call
0505 07:54:01.906873 (+1344927us) ts_tablet_manager.cc:1151] Deleting tablet e8bf49b787ef4de095448751464708a4
0505 07:54:01.906882 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.906989 (+ 107us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1336178}
I20260505 07:54:01.908226 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 444) took 1253 ms. Trace:
I20260505 07:54:01.908339 1178 rpcz_store.cc:276] 0505 07:54:00.655075 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.655624 (+ 549us) service_pool.cc:224] Handling call
0505 07:54:01.908067 (+1252443us) ts_tablet_manager.cc:1151] Deleting tablet feeafd614b3e4281a8e45b950e699b63
0505 07:54:01.908079 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.908176 (+ 97us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1252206}
I20260505 07:54:01.908697 1176 tablet_bootstrap.cc:492] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.908869 1176 ts_tablet_manager.cc:1403] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.002s
I20260505 07:54:01.909328 1172 tablet_bootstrap.cc:654] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.909765 1176 raft_consensus.cc:359] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.910012 1176 raft_consensus.cc:385] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.910105 1176 raft_consensus.cc:740] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.910386 1176 consensus_queue.cc:260] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.910893 1176 ts_tablet_manager.cc:1434] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:01.911217 1176 tablet_bootstrap.cc:492] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.911969 1178 tablet_replica.cc:333] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:01.912185 1178 raft_consensus.cc:2243] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:01.912396 1178 raft_consensus.cc:2272] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:01.912758 1176 tablet_bootstrap.cc:654] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.914649 1178 ts_tablet_manager.cc:1916] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:01.920356 1670 raft_consensus.cc:493] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 28d3eda04f9c47ae976d19c218b6dcc5)
I20260505 07:54:01.923995 1391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 2 candidate_status { last_received { term: 1 index: 589 } } ignore_live_leader: false dest_uuid: "23c57a2c927c4c54bccecfeb3a775311" is_pre_election: true
I20260505 07:54:01.926574 1670 raft_consensus.cc:515] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:54:01.928597 1172 tablet_bootstrap.cc:492] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.928825 1172 ts_tablet_manager.cc:1403] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.025s user 0.016s sys 0.002s
I20260505 07:54:01.928898 1391 raft_consensus.cc:2410] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 379cdfdecc3341ea9d7d9dc09274338d for term 2 because replica has last-logged OpId of term: 1 index: 590, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 589.
I20260505 07:54:01.929171 1176 tablet_bootstrap.cc:492] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.929322 1176 ts_tablet_manager.cc:1403] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.000s
I20260505 07:54:01.929787 1172 raft_consensus.cc:359] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.929980 1172 raft_consensus.cc:385] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.930050 1172 raft_consensus.cc:740] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.930300 1172 consensus_queue.cc:260] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.930289 1176 raft_consensus.cc:359] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:01.930519 1176 raft_consensus.cc:385] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.930631 1176 raft_consensus.cc:740] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.930874 1176 consensus_queue.cc:260] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:01.931442 1176 ts_tablet_manager.cc:1434] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:01.931751 1176 tablet_bootstrap.cc:492] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.932181 1172 ts_tablet_manager.cc:1434] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:01.932451 1172 tablet_bootstrap.cc:492] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.933563 1176 tablet_bootstrap.cc:654] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.935896 1172 tablet_bootstrap.cc:654] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.939429 1649 raft_consensus.cc:493] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.939648 1649 raft_consensus.cc:515] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.940438 1649 leader_election.cc:290] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:01.941951 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7219f9896fd94187ae714635077110af" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:01.942857 718 leader_election.cc:343] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 7219f9896fd94187ae714635077110af
W20260505 07:54:01.943239 717 leader_election.cc:343] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:54:01.943393 717 leader_election.cc:304] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:01.943972 1649 raft_consensus.cc:2749] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:01.944656 1172 tablet_bootstrap.cc:492] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:01.944787 1172 ts_tablet_manager.cc:1403] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.006s
I20260505 07:54:01.945536 1172 raft_consensus.cc:359] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.945712 1172 raft_consensus.cc:385] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.945811 1172 raft_consensus.cc:740] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.946031 1172 consensus_queue.cc:260] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.946179 1176 tablet_bootstrap.cc:492] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.946342 1176 ts_tablet_manager.cc:1403] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.002s
I20260505 07:54:01.947324 1176 raft_consensus.cc:359] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.947559 1176 raft_consensus.cc:385] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.947664 1176 raft_consensus.cc:740] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.947896 1172 ts_tablet_manager.cc:1434] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:01.947880 1176 consensus_queue.cc:260] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.948701 1176 ts_tablet_manager.cc:1434] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:01.949080 1176 tablet_bootstrap.cc:492] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.950382 1172 tablet_bootstrap.cc:492] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:01.951000 1176 tablet_bootstrap.cc:654] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.952574 1172 tablet_bootstrap.cc:654] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.958671 467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23c57a2c927c4c54bccecfeb3a775311" candidate_term: 2 candidate_status { last_received { term: 1 index: 590 } } ignore_live_leader: false dest_uuid: "379cdfdecc3341ea9d7d9dc09274338d" is_pre_election: true
I20260505 07:54:01.958964 467 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23c57a2c927c4c54bccecfeb3a775311 in term 1.
I20260505 07:54:01.960816 1317 leader_election.cc:304] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c57a2c927c4c54bccecfeb3a775311, 379cdfdecc3341ea9d7d9dc09274338d; no voters:
I20260505 07:54:01.964056 1649 raft_consensus.cc:493] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:01.964826 1176 tablet_bootstrap.cc:492] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.964819 1649 raft_consensus.cc:515] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.965026 1176 ts_tablet_manager.cc:1403] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260505 07:54:01.966125 1176 raft_consensus.cc:359] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.966315 1176 raft_consensus.cc:385] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.966410 1176 raft_consensus.cc:740] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.966606 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a79578757824bfc9df129e3475e7af3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:01.966665 1176 consensus_queue.cc:260] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.966881 1052 raft_consensus.cc:2468] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:01.967222 1176 ts_tablet_manager.cc:1434] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:01.967561 1176 tablet_bootstrap.cc:492] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:01.969350 1649 leader_election.cc:290] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:01.970449 718 leader_election.cc:343] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 4a79578757824bfc9df129e3475e7af3
I20260505 07:54:01.970927 717 leader_election.cc:304] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:01.974300 1670 leader_election.cc:290] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 379cdfdecc3341ea9d7d9dc09274338d (127.0.105.62:33621), 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401)
I20260505 07:54:01.975214 1649 raft_consensus.cc:2804] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:01.975406 1649 raft_consensus.cc:493] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:01.975528 1649 raft_consensus.cc:3060] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:01.979751 1670 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:01.980140 1670 raft_consensus.cc:493] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 28d3eda04f9c47ae976d19c218b6dcc5)
I20260505 07:54:01.980440 1670 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:01.980459 1181 tablet_bootstrap.cc:492] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:01.980683 1181 ts_tablet_manager.cc:1403] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.086s user 0.005s sys 0.000s
I20260505 07:54:01.981756 1181 raft_consensus.cc:359] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.982041 1181 raft_consensus.cc:385] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.982137 1181 raft_consensus.cc:740] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.982496 1181 consensus_queue.cc:260] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.990376 1181 ts_tablet_manager.cc:1434] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.010s user 0.003s sys 0.000s
I20260505 07:54:01.995055 1181 tablet_bootstrap.cc:492] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:01.995822 1176 tablet_bootstrap.cc:654] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:01.998147 1176 tablet_bootstrap.cc:492] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:01.998319 1176 ts_tablet_manager.cc:1403] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.000s
I20260505 07:54:01.998864 1649 raft_consensus.cc:515] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:01.999157 1176 raft_consensus.cc:359] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:01.999357 1176 raft_consensus.cc:385] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:01.999445 1176 raft_consensus.cc:740] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:01.999701 1176 consensus_queue.cc:260] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.000053 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a79578757824bfc9df129e3475e7af3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:02.000263 1176 ts_tablet_manager.cc:1434] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:02.000262 1052 raft_consensus.cc:3060] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.000586 1176 tablet_bootstrap.cc:492] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.002324 1176 tablet_bootstrap.cc:654] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.003121 1052 raft_consensus.cc:2468] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:02.007829 717 leader_election.cc:304] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.008225 1512 raft_consensus.cc:2804] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.008401 1512 raft_consensus.cc:697] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:02.008724 1512 consensus_queue.cc:237] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:02.011935 1178 ts_tablet_manager.cc:1929] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:02.012076 1178 log.cc:1199] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/7219f9896fd94187ae714635077110af
I20260505 07:54:02.012588 1178 ts_tablet_manager.cc:1950] T 7219f9896fd94187ae714635077110af P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:02.013489 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 446) took 1358 ms. Trace:
I20260505 07:54:02.013662 1178 rpcz_store.cc:276] 0505 07:54:00.655481 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.655755 (+ 274us) service_pool.cc:224] Handling call
0505 07:54:01.911893 (+1256138us) ts_tablet_manager.cc:1151] Deleting tablet 7219f9896fd94187ae714635077110af
0505 07:54:01.911901 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:01.976694 (+ 64793us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:02.011827 (+ 35133us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:02.013441 (+ 1614us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1255942}
W20260505 07:54:02.014818 718 leader_election.cc:343] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 4a79578757824bfc9df129e3475e7af3
I20260505 07:54:02.015100 1649 leader_election.cc:290] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.015707 1670 raft_consensus.cc:515] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:54:02.017813 1670 leader_election.cc:290] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [CANDIDATE]: Term 2 election: Requested vote from peers 379cdfdecc3341ea9d7d9dc09274338d (127.0.105.62:33621), 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401)
I20260505 07:54:02.018391 467 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23c57a2c927c4c54bccecfeb3a775311" candidate_term: 2 candidate_status { last_received { term: 1 index: 590 } } ignore_live_leader: false dest_uuid: "379cdfdecc3341ea9d7d9dc09274338d"
I20260505 07:54:02.018651 467 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:02.019665 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 445) took 1364 ms. Trace:
I20260505 07:54:02.019780 1178 rpcz_store.cc:276] 0505 07:54:00.655354 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.655635 (+ 281us) service_pool.cc:224] Handling call
0505 07:54:02.019444 (+1363809us) ts_tablet_manager.cc:1151] Deleting tablet 27bacc97d7534284970f218cf68ac75b
0505 07:54:02.019455 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:02.019633 (+ 178us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1363328}
I20260505 07:54:02.019922 1172 tablet_bootstrap.cc:492] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.020095 1172 ts_tablet_manager.cc:1403] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.072s user 0.023s sys 0.029s
I20260505 07:54:02.020959 1172 raft_consensus.cc:359] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.021142 1172 raft_consensus.cc:385] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.021207 1172 raft_consensus.cc:740] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.021430 1172 consensus_queue.cc:260] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.021672 467 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23c57a2c927c4c54bccecfeb3a775311 in term 2.
I20260505 07:54:02.023386 1172 ts_tablet_manager.cc:1434] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:02.023708 1172 tablet_bootstrap.cc:492] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.023870 1317 leader_election.cc:304] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23c57a2c927c4c54bccecfeb3a775311, 379cdfdecc3341ea9d7d9dc09274338d; no voters:
I20260505 07:54:02.024188 1670 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:02.024567 1670 raft_consensus.cc:697] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [term 2 LEADER]: Becoming Leader. State: Replica: 23c57a2c927c4c54bccecfeb3a775311, State: Running, Role: LEADER
I20260505 07:54:02.025106 1670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 589, Committed index: 589, Last appended: 1.590, Last appended by leader: 590, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:54:02.023835 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 447) took 1364 ms. Trace:
I20260505 07:54:02.025420 1172 tablet_bootstrap.cc:654] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.025394 1178 rpcz_store.cc:276] 0505 07:54:00.659249 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.659298 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:02.023694 (+1364396us) ts_tablet_manager.cc:1151] Deleting tablet 60772a9697ba417b879dc6025eeb8eec
0505 07:54:02.023705 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:02.023802 (+ 97us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1351743}
I20260505 07:54:02.026578 1178 tablet_replica.cc:333] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:02.029177 1178 raft_consensus.cc:2243] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:02.030624 1178 raft_consensus.cc:2272] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:02.031266 1178 ts_tablet_manager.cc:1916] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:02.031344 1172 tablet_bootstrap.cc:492] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.031478 1172 ts_tablet_manager.cc:1403] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260505 07:54:02.031783 1679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23c57a2c927c4c54bccecfeb3a775311. Latest consensus state: current_term: 2 leader_uuid: "23c57a2c927c4c54bccecfeb3a775311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:54:02.031991 1679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: This master's current role is: LEADER
I20260505 07:54:02.033032 1172 raft_consensus.cc:359] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.033200 1172 raft_consensus.cc:385] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.033289 1172 raft_consensus.cc:740] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.033483 1172 consensus_queue.cc:260] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.035532 1172 ts_tablet_manager.cc:1434] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:02.035835 1172 tablet_bootstrap.cc:492] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.037432 1172 tablet_bootstrap.cc:654] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.042501 1172 tablet_bootstrap.cc:492] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.042649 1172 ts_tablet_manager.cc:1403] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260505 07:54:02.043346 1172 raft_consensus.cc:359] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.043540 1172 raft_consensus.cc:385] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.043637 1172 raft_consensus.cc:740] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.045394 1172 consensus_queue.cc:260] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.046360 1172 ts_tablet_manager.cc:1434] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:02.046674 1172 tablet_bootstrap.cc:492] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.048844 1172 tablet_bootstrap.cc:654] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.052269 1567 raft_consensus.cc:493] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.052438 1567 raft_consensus.cc:515] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:02.053187 1567 leader_election.cc:290] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.054256 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd36b5a84b4442fb76e1610d8af4c07" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:02.054486 919 raft_consensus.cc:2468] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.055066 991 leader_election.cc:304] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:02.055779 1567 raft_consensus.cc:2804] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.055817 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd36b5a84b4442fb76e1610d8af4c07" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.055896 1567 raft_consensus.cc:493] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.056015 1567 raft_consensus.cc:3060] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.056015 781 raft_consensus.cc:2468] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.059285 1567 raft_consensus.cc:515] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:02.059446 1172 tablet_bootstrap.cc:492] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.059561 1172 ts_tablet_manager.cc:1403] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.003s
I20260505 07:54:02.060287 1172 raft_consensus.cc:359] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:02.060461 1172 raft_consensus.cc:385] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.060443 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd36b5a84b4442fb76e1610d8af4c07" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.060539 1172 raft_consensus.cc:740] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.060626 781 raft_consensus.cc:3060] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.060760 1172 consensus_queue.cc:260] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:02.061022 1178 ts_tablet_manager.cc:1929] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:02.061173 1178 log.cc:1199] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/4a79578757824bfc9df129e3475e7af3
I20260505 07:54:02.061551 1178 ts_tablet_manager.cc:1950] T 4a79578757824bfc9df129e3475e7af3 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:02.062386 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 448) took 1302 ms. Trace:
I20260505 07:54:02.062500 1178 rpcz_store.cc:276] 0505 07:54:00.759878 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.828127 (+ 68249us) service_pool.cc:224] Handling call
0505 07:54:02.026503 (+1198376us) ts_tablet_manager.cc:1151] Deleting tablet 4a79578757824bfc9df129e3475e7af3
0505 07:54:02.026512 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:02.040849 (+ 14337us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:02.060950 (+ 20101us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:02.062330 (+ 1380us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1150204}
I20260505 07:54:02.063959 1512 raft_consensus.cc:493] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.064034 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 452) took 1222 ms. Trace:
I20260505 07:54:02.064097 1512 raft_consensus.cc:515] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.064132 1178 rpcz_store.cc:276] 0505 07:54:00.842032 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.854206 (+ 12174us) service_pool.cc:224] Handling call
0505 07:54:02.063852 (+1209646us) ts_tablet_manager.cc:1151] Deleting tablet e5eaa91ecc1f43d18b313b6be4cc211e
0505 07:54:02.063865 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:02.063979 (+ 114us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1185080}
I20260505 07:54:02.064867 1512 leader_election.cc:290] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.064932 1172 ts_tablet_manager.cc:1434] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260505 07:54:02.065060 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 453) took 1212 ms. Trace:
I20260505 07:54:02.065164 1178 rpcz_store.cc:276] 0505 07:54:00.852241 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.858789 (+ 6548us) service_pool.cc:224] Handling call
0505 07:54:02.064888 (+1206099us) ts_tablet_manager.cc:1151] Deleting tablet 9a06f77255544d7dabb71a4f70ada809
0505 07:54:02.064900 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:02.065003 (+ 103us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1182331}
I20260505 07:54:02.065850 1172 tablet_bootstrap.cc:492] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:02.066222 718 leader_election.cc:343] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 3d6494590052459485059f5dce46f34f
I20260505 07:54:02.067833 1172 tablet_bootstrap.cc:654] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.067885 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 456) took 1174 ms. Trace:
I20260505 07:54:02.068017 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6494590052459485059f5dce46f34f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.068281 1052 raft_consensus.cc:2468] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:02.067986 1178 rpcz_store.cc:276] 0505 07:54:00.893362 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.893412 (+ 50us) service_pool.cc:224] Handling call
0505 07:54:02.067705 (+1174293us) ts_tablet_manager.cc:1151] Deleting tablet 080b2984fcf2478a8def6540da63554d
0505 07:54:02.067714 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:02.067836 (+ 122us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1174001}
I20260505 07:54:02.069818 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcd36b5a84b4442fb76e1610d8af4c07" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:02.070081 919 raft_consensus.cc:3060] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.072055 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 455) took 1204 ms. Trace:
I20260505 07:54:02.072153 1178 rpcz_store.cc:276] 0505 07:54:00.865240 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:00.865322 (+ 82us) service_pool.cc:224] Handling call
0505 07:54:02.069886 (+1204564us) ts_tablet_manager.cc:1151] Deleting tablet 09b46ff0101a4e529077b9e30492ace7
0505 07:54:02.069895 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:02.069967 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1173733}
I20260505 07:54:02.072593 1178 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45248 (request call id 458) took 1054 ms. Trace:
I20260505 07:54:02.072669 1178 rpcz_store.cc:276] 0505 07:54:01.018560 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:01.019341 (+ 781us) service_pool.cc:224] Handling call
0505 07:54:02.072435 (+1053094us) ts_tablet_manager.cc:1151] Deleting tablet aab3850799254dda9716fd5bf9404159
0505 07:54:02.072444 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:02.072516 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1046884}
I20260505 07:54:02.073835 919 raft_consensus.cc:2468] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.074378 991 leader_election.cc:304] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:02.074676 1596 raft_consensus.cc:2804] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.074822 1567 leader_election.cc:290] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.074856 1596 raft_consensus.cc:697] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:02.075102 1596 consensus_queue.cc:237] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:02.080812 781 raft_consensus.cc:2468] T fcd36b5a84b4442fb76e1610d8af4c07 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.080844 717 leader_election.cc:304] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:02.081774 1512 raft_consensus.cc:2804] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.081910 1512 raft_consensus.cc:493] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.082010 1512 raft_consensus.cc:3060] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.086045 1512 raft_consensus.cc:515] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.086809 1567 raft_consensus.cc:493] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.087081 1567 raft_consensus.cc:515] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.089216 1181 tablet_bootstrap.cc:654] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.096500 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "976359601ad74b149696dc7f555b550c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.096771 781 raft_consensus.cc:2468] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.097389 988 leader_election.cc:304] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.097711 1596 raft_consensus.cc:2804] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.097822 1596 raft_consensus.cc:493] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.097936 1567 leader_election.cc:290] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.097937 1596 raft_consensus.cc:3060] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.100780 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d6494590052459485059f5dce46f34f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:02.101095 1052 raft_consensus.cc:3060] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.101123 1596 raft_consensus.cc:515] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.102037 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "976359601ad74b149696dc7f555b550c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.102208 781 raft_consensus.cc:3060] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.105347 781 raft_consensus.cc:2468] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.105545 1052 raft_consensus.cc:2468] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:02.105831 988 leader_election.cc:304] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.106122 717 leader_election.cc:304] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.106470 1649 raft_consensus.cc:2804] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.106814 1649 raft_consensus.cc:697] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
W20260505 07:54:02.106952 991 leader_election.cc:343] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 976359601ad74b149696dc7f555b550c
W20260505 07:54:02.107239 991 leader_election.cc:343] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 976359601ad74b149696dc7f555b550c
I20260505 07:54:02.107210 1649 consensus_queue.cc:237] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.107388 1596 leader_election.cc:290] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.107676 1596 raft_consensus.cc:2804] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.107897 1596 raft_consensus.cc:697] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:02.108173 1596 consensus_queue.cc:237] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:02.110270 718 leader_election.cc:343] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 3d6494590052459485059f5dce46f34f
I20260505 07:54:02.110574 1512 leader_election.cc:290] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.125281 1181 tablet_bootstrap.cc:492] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.125451 1181 ts_tablet_manager.cc:1403] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.131s user 0.008s sys 0.005s
I20260505 07:54:02.126403 1181 raft_consensus.cc:359] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.126614 1181 raft_consensus.cc:385] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.126703 1181 raft_consensus.cc:740] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.127710 1176 tablet_bootstrap.cc:492] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.127882 1176 ts_tablet_manager.cc:1403] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.127s user 0.032s sys 0.022s
I20260505 07:54:02.128803 1176 raft_consensus.cc:359] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.129005 1176 raft_consensus.cc:385] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.129113 1176 raft_consensus.cc:740] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.129345 1176 consensus_queue.cc:260] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.129863 1176 ts_tablet_manager.cc:1434] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:02.130052 1181 consensus_queue.cc:260] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.132318 1181 ts_tablet_manager.cc:1434] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:54:02.132637 1181 tablet_bootstrap.cc:492] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.133147 1176 tablet_bootstrap.cc:492] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.133505 1596 raft_consensus.cc:493] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.133648 1596 raft_consensus.cc:515] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.134176 1596 leader_election.cc:290] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.134830 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909306e1e5f34b7c8fbc0d86cafce531" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.135010 781 raft_consensus.cc:2468] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.135408 1176 tablet_bootstrap.cc:654] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.135898 988 leader_election.cc:304] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.136318 1567 raft_consensus.cc:2804] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.136399 1567 raft_consensus.cc:493] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.136503 1567 raft_consensus.cc:3060] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.139164 1567 raft_consensus.cc:515] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.140272 1569 raft_consensus.cc:493] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.140447 1569 raft_consensus.cc:515] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.141014 1569 leader_election.cc:290] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:02.141690 991 leader_election.cc:343] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 909306e1e5f34b7c8fbc0d86cafce531
I20260505 07:54:02.142637 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "909306e1e5f34b7c8fbc0d86cafce531" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.142797 782 raft_consensus.cc:3060] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.144873 1172 tablet_bootstrap.cc:492] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.145031 1172 ts_tablet_manager.cc:1403] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.080s user 0.011s sys 0.009s
I20260505 07:54:02.145606 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accb2dedc30e494ebd22c33594cee661" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.146126 1652 consensus_queue.cc:1048] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.146252 1172 raft_consensus.cc:359] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.146423 1172 raft_consensus.cc:385] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.146486 1172 raft_consensus.cc:740] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.146664 1567 leader_election.cc:290] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.146692 1172 consensus_queue.cc:260] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.147158 1172 ts_tablet_manager.cc:1434] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260505 07:54:02.147840 856 leader_election.cc:343] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.148109 991 leader_election.cc:343] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 909306e1e5f34b7c8fbc0d86cafce531
I20260505 07:54:02.149571 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accb2dedc30e494ebd22c33594cee661" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.150012 854 leader_election.cc:343] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.150133 854 leader_election.cc:304] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.150566 1569 raft_consensus.cc:2749] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.151692 1172 tablet_bootstrap.cc:492] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.153053 1181 tablet_bootstrap.cc:654] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.154987 1172 tablet_bootstrap.cc:654] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.162465 1596 raft_consensus.cc:493] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.162647 1596 raft_consensus.cc:515] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.163218 1596 leader_election.cc:290] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:02.164657 991 leader_election.cc:343] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 7e707143a4e64725aa66db05b24999a6
I20260505 07:54:02.165916 1181 tablet_bootstrap.cc:492] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.166178 1181 ts_tablet_manager.cc:1403] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.034s user 0.002s sys 0.003s
I20260505 07:54:02.166723 782 raft_consensus.cc:2468] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.167129 1181 raft_consensus.cc:359] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.167940 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e707143a4e64725aa66db05b24999a6" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.168159 1181 raft_consensus.cc:385] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.168304 1181 raft_consensus.cc:740] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.168387 988 leader_election.cc:304] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:02.168676 1181 consensus_queue.cc:260] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:02.168813 988 leader_election.cc:343] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.171686 988 leader_election.cc:304] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.168963 1596 raft_consensus.cc:2804] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.172245 1596 raft_consensus.cc:697] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:02.172565 1596 consensus_queue.cc:237] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.173753 1652 raft_consensus.cc:2749] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.176470 1567 raft_consensus.cc:493] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.176718 1567 raft_consensus.cc:515] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.177920 1649 raft_consensus.cc:493] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.178066 1649 raft_consensus.cc:515] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.178695 1649 leader_election.cc:290] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:02.178712 991 leader_election.cc:343] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 202a237ca580482c98803683a77d96d2
I20260505 07:54:02.178974 1567 leader_election.cc:290] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.180371 1176 tablet_bootstrap.cc:492] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.180523 1176 ts_tablet_manager.cc:1403] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.047s user 0.005s sys 0.000s
I20260505 07:54:02.181248 1176 raft_consensus.cc:359] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.181412 1176 raft_consensus.cc:385] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.181473 1176 raft_consensus.cc:740] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.181668 1176 consensus_queue.cc:260] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.182130 1176 ts_tablet_manager.cc:1434] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:02.182130 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872362aecf6a44b59e586b1fa7637c8b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.182312 1052 raft_consensus.cc:2468] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:02.182435 1176 tablet_bootstrap.cc:492] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.184388 717 leader_election.cc:304] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.184867 1512 raft_consensus.cc:2804] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.184966 1512 raft_consensus.cc:493] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.185079 1512 raft_consensus.cc:3060] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.186293 1181 ts_tablet_manager.cc:1434] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.020s user 0.001s sys 0.001s
I20260505 07:54:02.186564 1181 tablet_bootstrap.cc:492] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.188390 1181 tablet_bootstrap.cc:654] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.190531 1176 tablet_bootstrap.cc:654] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.193883 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "202a237ca580482c98803683a77d96d2" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.194054 781 raft_consensus.cc:2468] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:02.194377 718 leader_election.cc:343] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 872362aecf6a44b59e586b1fa7637c8b
I20260505 07:54:02.194751 988 leader_election.cc:304] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:02.197782 1580 raft_consensus.cc:2804] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.197872 1580 raft_consensus.cc:493] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.197962 1580 raft_consensus.cc:3060] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.200754 1580 raft_consensus.cc:515] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.201484 1580 leader_election.cc:290] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.201635 1580 consensus_queue.cc:1048] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.202222 1512 raft_consensus.cc:515] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.202771 1512 leader_election.cc:290] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.203342 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "872362aecf6a44b59e586b1fa7637c8b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:02.203517 1052 raft_consensus.cc:3060] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.204660 1649 raft_consensus.cc:493] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.204838 1649 raft_consensus.cc:515] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.205466 1649 leader_election.cc:290] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.206192 1693 raft_consensus.cc:493] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.206353 1693 raft_consensus.cc:515] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.206897 1052 raft_consensus.cc:2468] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:02.207033 1693 leader_election.cc:290] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.207875 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d485ae2d1d4d76a6a5d988cec8291a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:02.208369 856 leader_election.cc:343] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.209645 991 leader_election.cc:343] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 202a237ca580482c98803683a77d96d2
I20260505 07:54:02.211930 1596 raft_consensus.cc:493] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.212072 1596 raft_consensus.cc:515] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.212363 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e707143a4e64725aa66db05b24999a6" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.212555 1052 raft_consensus.cc:2468] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:02.212857 1596 leader_election.cc:290] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.213617 1172 tablet_bootstrap.cc:492] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
W20260505 07:54:02.213646 991 leader_election.cc:343] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 1838f89dc14a47f99f95f4cc5fdefd70
I20260505 07:54:02.213747 1172 ts_tablet_manager.cc:1403] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.062s user 0.009s sys 0.012s
I20260505 07:54:02.213861 717 leader_election.cc:304] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.214003 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "202a237ca580482c98803683a77d96d2" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.214609 1172 raft_consensus.cc:359] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.214782 1172 raft_consensus.cc:385] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.214900 1172 raft_consensus.cc:740] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.215137 1172 consensus_queue.cc:260] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.215813 1172 ts_tablet_manager.cc:1434] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260505 07:54:02.216163 1172 tablet_bootstrap.cc:492] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.217986 717 leader_election.cc:304] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.219010 1172 tablet_bootstrap.cc:654] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.222988 1181 tablet_bootstrap.cc:492] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
W20260505 07:54:02.225651 718 leader_election.cc:343] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 872362aecf6a44b59e586b1fa7637c8b
I20260505 07:54:02.225997 782 raft_consensus.cc:3060] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:02.226683 718 leader_election.cc:343] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 7e707143a4e64725aa66db05b24999a6
I20260505 07:54:02.227145 1512 raft_consensus.cc:2804] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.227694 1649 raft_consensus.cc:2804] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.228298 1649 raft_consensus.cc:493] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.228443 1649 raft_consensus.cc:3060] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.229233 1693 raft_consensus.cc:493] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.229367 1693 raft_consensus.cc:515] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.229889 1693 leader_election.cc:290] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.228060 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d485ae2d1d4d76a6a5d988cec8291a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.230976 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e278b86e7b5d444ebad8d0f6921d4c62" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.231442 854 leader_election.cc:343] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.231652 854 leader_election.cc:304] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:02.231971 854 leader_election.cc:343] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.232230 1693 raft_consensus.cc:2749] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.233747 782 raft_consensus.cc:2468] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.235191 1649 raft_consensus.cc:515] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.235514 1512 raft_consensus.cc:697] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:02.236266 1512 consensus_queue.cc:237] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.235663 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1838f89dc14a47f99f95f4cc5fdefd70" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.237464 779 raft_consensus.cc:2468] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.235972 1176 tablet_bootstrap.cc:492] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.239075 1176 ts_tablet_manager.cc:1403] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.057s user 0.008s sys 0.000s
I20260505 07:54:02.244004 1176 raft_consensus.cc:359] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:02.244360 718 leader_election.cc:343] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 7e707143a4e64725aa66db05b24999a6
I20260505 07:54:02.244477 1649 leader_election.cc:290] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.245137 1176 raft_consensus.cc:385] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.245420 1176 raft_consensus.cc:740] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.245968 1176 consensus_queue.cc:260] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.247117 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 909306e1e5f34b7c8fbc0d86cafce531 (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]) successfully deleted
I20260505 07:54:02.247612 1176 ts_tablet_manager.cc:1434] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:54:02.248011 1176 tablet_bootstrap.cc:492] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:54:02.251641 585 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f84f464e83cb412eade9978ea99f6685 consensus state has no leader
W20260505 07:54:02.252014 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e8bf49b787ef4de095448751464708a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8bf49b787ef4de095448751464708a4 already in progress: creating tablet
W20260505 07:54:02.252271 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 27bacc97d7534284970f218cf68ac75b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 27bacc97d7534284970f218cf68ac75b already in progress: creating tablet
W20260505 07:54:02.252468 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet feeafd614b3e4281a8e45b950e699b63 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet feeafd614b3e4281a8e45b950e699b63 already in progress: creating tablet
I20260505 07:54:02.252637 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 7219f9896fd94187ae714635077110af (table default.Table_ad09d6b42bc344c6ba96f4322d75601f [id=d82da77797e14074b53f00e8e741744c]) successfully deleted
I20260505 07:54:02.230410 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e278b86e7b5d444ebad8d0f6921d4c62" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:02.252867 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 60772a9697ba417b879dc6025eeb8eec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60772a9697ba417b879dc6025eeb8eec already in progress: creating tablet
I20260505 07:54:02.253014 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 1838f89dc14a47f99f95f4cc5fdefd70 (table default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958]) successfully deleted
I20260505 07:54:02.234287 988 leader_election.cc:304] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:02.253055 1181 ts_tablet_manager.cc:1403] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.067s user 0.008s sys 0.004s
W20260505 07:54:02.253233 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e5eaa91ecc1f43d18b313b6be4cc211e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5eaa91ecc1f43d18b313b6be4cc211e already in progress: creating tablet
W20260505 07:54:02.253468 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 09b46ff0101a4e529077b9e30492ace7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09b46ff0101a4e529077b9e30492ace7 already in progress: creating tablet
W20260505 07:54:02.253662 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 080b2984fcf2478a8def6540da63554d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 080b2984fcf2478a8def6540da63554d already in progress: creating tablet
I20260505 07:54:02.253647 988 leader_election.cc:304] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:02.253821 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 3d6494590052459485059f5dce46f34f (table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f]) successfully deleted
W20260505 07:54:02.254032 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet aab3850799254dda9716fd5bf9404159 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet aab3850799254dda9716fd5bf9404159 already in progress: creating tablet
W20260505 07:54:02.254120 856 leader_election.cc:343] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.254215 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 6546745e894545ff9c5f91bcc6e1a41e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6546745e894545ff9c5f91bcc6e1a41e already in progress: creating tablet
I20260505 07:54:02.254266 856 leader_election.cc:304] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.254384 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 7e707143a4e64725aa66db05b24999a6 (table default.Table_f0316c88590245aea8815b3b08e89f45 [id=c3a480b747de444aa45b771244908352]) successfully deleted
W20260505 07:54:02.254547 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet f2f5b72f58a24db0b58c15b47227bc60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2f5b72f58a24db0b58c15b47227bc60 already in progress: creating tablet
I20260505 07:54:02.254665 1693 raft_consensus.cc:2749] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.254733 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 1ef4f516e64a44fd8cd33674fb808067 (table default.Table_829cc346ac834637ab2e2198944a8550 [id=2c91868ddba84103b9a5c78201842a9e]) successfully deleted
I20260505 07:54:02.240319 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e707143a4e64725aa66db05b24999a6" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:02.254915 588 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet b0b9e8eb767a4eceb583ff9b47000d58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0b9e8eb767a4eceb583ff9b47000d58 already in progress: creating tablet
I20260505 07:54:02.254921 1055 raft_consensus.cc:3060] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.255092 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet bd581a6799a14a8dafb80d411e8936f3 (table default.Table_822258eaa3b14528a8bfa97e3e9d4d7a [id=76b0eb99a1f343829e5ba35de6302cf2]) successfully deleted
I20260505 07:54:02.255268 588 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet bf76e868bbcb44b0b1887c84dd27f093 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]) successfully deleted
I20260505 07:54:02.255421 600 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47238:
name: "default.Table_89c343e38c814e91b9aa1da388b52770"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:02.257339 1176 tablet_bootstrap.cc:654] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.257663 1055 raft_consensus.cc:2468] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:02.258162 717 leader_election.cc:304] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:02.258431 1649 raft_consensus.cc:2804] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.258731 1649 raft_consensus.cc:697] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:02.258776 1596 raft_consensus.cc:2804] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.258926 1596 raft_consensus.cc:697] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:02.259136 1649 consensus_queue.cc:237] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.259217 1596 consensus_queue.cc:237] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.260061 1693 raft_consensus.cc:493] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.260191 1693 raft_consensus.cc:515] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.260877 1693 leader_election.cc:290] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.261656 1580 raft_consensus.cc:2804] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.261770 1580 raft_consensus.cc:493] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.261873 1580 raft_consensus.cc:3060] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:02.258482 631 rpcz_store.cc:267] Call kudu.server.GenericService.CheckLeaks from 127.0.0.1:47188 (request call id 22) took 874 ms (client timeout 1000 ms). Trace:
W20260505 07:54:02.263862 600 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89c343e38c814e91b9aa1da388b52770 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:02.263999 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet c7652191b1d44ba59d856a2b0d7017b1 (table default.Table_6e9d636b8b314e5b888c725d34775180 [id=73c0bcbf3485405589f58b8175bfc3b9]) successfully deleted
I20260505 07:54:02.264039 1181 raft_consensus.cc:359] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.264276 1181 raft_consensus.cc:385] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.264421 1181 raft_consensus.cc:740] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.264433 610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23c57a2c927c4c54bccecfeb3a775311" candidate_term: 2 candidate_status { last_received { term: 1 index: 590 } } ignore_live_leader: false dest_uuid: "28d3eda04f9c47ae976d19c218b6dcc5"
I20260505 07:54:02.264451 1580 raft_consensus.cc:515] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.264758 611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "23c57a2c927c4c54bccecfeb3a775311" candidate_term: 2 candidate_status { last_received { term: 1 index: 590 } } ignore_live_leader: false dest_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" is_pre_election: true
I20260505 07:54:02.264685 1181 consensus_queue.cc:260] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.265615 1580 leader_election.cc:290] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.266319 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1838f89dc14a47f99f95f4cc5fdefd70" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.266496 779 raft_consensus.cc:3060] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.267983 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d878ae0ddb42319560f74e0a490321" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:02.268159 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 727cacd242844455915bd9963d2db19f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 727cacd242844455915bd9963d2db19f already in progress: creating tablet
W20260505 07:54:02.268384 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d86b8b4dadf4fd2b6409a8cb70e313e already in progress: creating tablet
W20260505 07:54:02.268510 856 leader_election.cc:343] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.269184 991 leader_election.cc:343] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 1838f89dc14a47f99f95f4cc5fdefd70
W20260505 07:54:02.271054 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e29ba55a846548679cb3b86305d0d6ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e29ba55a846548679cb3b86305d0d6ed already in progress: creating tablet
W20260505 07:54:02.271343 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7001482d90364dbdb8b9c5bfb98e1cd4 already in progress: creating tablet
W20260505 07:54:02.272536 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet d7e0c8ad716e443588d4ae3aa28ed4bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d7e0c8ad716e443588d4ae3aa28ed4bf already in progress: creating tablet
I20260505 07:54:02.272776 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet c6dcbebddcdc4ad9a788c01134e1101c (table default.Table_ba04ccad67434c7884329b23efea4bfd [id=36cc8dc1683b4ef194ec299e4fed4de9]) successfully deleted
I20260505 07:54:02.272749 779 raft_consensus.cc:2468] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.273253 988 leader_election.cc:304] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:02.273733 1567 raft_consensus.cc:2804] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.273952 1567 raft_consensus.cc:697] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:02.274430 1567 consensus_queue.cc:237] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:02.274680 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8b406c4c6ba741bc85eef54c6f576459 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b406c4c6ba741bc85eef54c6f576459 already in progress: creating tablet
W20260505 07:54:02.274936 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet a7d8265c05b74b28959d0de41fad1fde because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7d8265c05b74b28959d0de41fad1fde already in progress: creating tablet
I20260505 07:54:02.275121 611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 2 candidate_status { last_received { term: 1 index: 589 } } ignore_live_leader: false dest_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" is_pre_election: true
I20260505 07:54:02.275769 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d878ae0ddb42319560f74e0a490321" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.276015 782 raft_consensus.cc:2468] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:02.276422 854 leader_election.cc:304] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
W20260505 07:54:02.277096 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 02412333633a4cdebe95a144fa6c363c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02412333633a4cdebe95a144fa6c363c already in progress: creating tablet
I20260505 07:54:02.277130 1693 raft_consensus.cc:2804] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.277231 1693 raft_consensus.cc:493] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.277320 1693 raft_consensus.cc:3060] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:02.277326 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 067c32ecccb44bca8bf9cc3f6c229326 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 067c32ecccb44bca8bf9cc3f6c229326 already in progress: creating tablet
I20260505 07:54:02.279240 1569 raft_consensus.cc:493] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:02.279259 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 3075e87e078f4d2aa1d329d18902354b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3075e87e078f4d2aa1d329d18902354b already in progress: creating tablet
I20260505 07:54:02.279438 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 25a74b65d5d6459293ab4461f4d34cf2 (table default.Table_eb39510acd7a40b6a7f20bd09e300ff7 [id=94a6d460807e43768c78bc319239b699]) successfully deleted
I20260505 07:54:02.279394 1569 raft_consensus.cc:515] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.280007 1569 leader_election.cc:290] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:02.281136 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 4f3e076892234c1494b1a5c13cbfd2e2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f3e076892234c1494b1a5c13cbfd2e2 already in progress: creating tablet
W20260505 07:54:02.281335 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 0459f2d45a2d44f1a73e0a0fff53866c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0459f2d45a2d44f1a73e0a0fff53866c already in progress: creating tablet
I20260505 07:54:02.281656 1693 raft_consensus.cc:515] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.282182 1693 leader_election.cc:290] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.283115 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d878ae0ddb42319560f74e0a490321" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:02.283771 856 leader_election.cc:343] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.284734 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9044763bed1a41739ecce4def4cca1fe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.284741 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d878ae0ddb42319560f74e0a490321" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.285019 779 raft_consensus.cc:3060] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:02.285218 856 leader_election.cc:343] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.285454 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9044763bed1a41739ecce4def4cca1fe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.285912 854 leader_election.cc:343] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.286039 854 leader_election.cc:304] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.286366 1693 raft_consensus.cc:2749] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.286568 1580 raft_consensus.cc:493] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.286707 1580 raft_consensus.cc:515] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.288339 1580 leader_election.cc:290] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.288511 1580 raft_consensus.cc:493] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.288810 1580 raft_consensus.cc:515] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.289264 779 raft_consensus.cc:2468] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
W20260505 07:54:02.289538 587 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f121fdd001ec4ea38ccd24a6afd9e51c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f121fdd001ec4ea38ccd24a6afd9e51c already in progress: creating tablet
I20260505 07:54:02.289755 854 leader_election.cc:304] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:02.289983 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ef4f516e64a44fd8cd33674fb808067" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.290246 1693 raft_consensus.cc:2804] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.290436 1693 raft_consensus.cc:697] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
W20260505 07:54:02.289916 631 rpcz_store.cc:269] 0505 07:54:01.375983 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:01.376046 (+ 63us) service_pool.cc:224] Handling call
0505 07:54:02.249938 (+873892us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:02.290791 1693 consensus_queue.cc:237] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:02.291231 988 leader_election.cc:343] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.291642 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d113a872f9400c8f28d0239ca42377" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.291867 782 raft_consensus.cc:2468] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.292343 988 leader_election.cc:304] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.292802 587 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 3d83e9d4243e409794ffe8a342bce6b3 (table default.Table_563025047e4142d4807295cff836ea99 [id=5a7033e8ecc940e49794281e421153ea]) successfully deleted
I20260505 07:54:02.292832 444 leader_election.cc:304] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 379cdfdecc3341ea9d7d9dc09274338d; no voters: 23c57a2c927c4c54bccecfeb3a775311, 28d3eda04f9c47ae976d19c218b6dcc5
I20260505 07:54:02.293009 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu with pid 567
I20260505 07:54:02.293310 1668 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:02.294569 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d113a872f9400c8f28d0239ca42377" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:02.294930 919 raft_consensus.cc:2468] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.296089 1580 leader_election.cc:290] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.296291 1580 raft_consensus.cc:2804] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.296487 1580 raft_consensus.cc:493] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.296671 1580 raft_consensus.cc:3060] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.299824 1580 raft_consensus.cc:515] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.309031 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d113a872f9400c8f28d0239ca42377" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.309191 781 raft_consensus.cc:3060] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:02.300510 991 leader_election.cc:343] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 1ef4f516e64a44fd8cd33674fb808067
I20260505 07:54:02.311287 991 leader_election.cc:304] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.311823 1580 leader_election.cc:290] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.311990 1580 raft_consensus.cc:2749] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.312842 1717 tablet.cc:1724] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:02.313125 1181 ts_tablet_manager.cc:1434] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.060s user 0.001s sys 0.002s
I20260505 07:54:02.313504 1181 tablet_bootstrap.cc:492] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.314296 1172 tablet_bootstrap.cc:492] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.314438 1172 ts_tablet_manager.cc:1403] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.098s user 0.016s sys 0.067s
I20260505 07:54:02.315119 1181 tablet_bootstrap.cc:654] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.315375 1172 raft_consensus.cc:359] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.315619 1172 raft_consensus.cc:385] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.315694 1172 raft_consensus.cc:740] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.315908 1172 consensus_queue.cc:260] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.316372 1172 ts_tablet_manager.cc:1434] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:02.316664 1172 tablet_bootstrap.cc:492] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.318210 1172 tablet_bootstrap.cc:654] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.319921 781 raft_consensus.cc:2468] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.320345 988 leader_election.cc:304] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.322836 1567 raft_consensus.cc:2804] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.323173 1567 raft_consensus.cc:697] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:02.323516 1567 consensus_queue.cc:237] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.338246 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5d113a872f9400c8f28d0239ca42377" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:02.338549 919 raft_consensus.cc:3060] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.341765 1716 tablet.cc:1724] T 571257bbb7e94315b60d84d6aa4917b1 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:02.342325 919 raft_consensus.cc:2468] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.346978 1693 raft_consensus.cc:493] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.347949 1693 raft_consensus.cc:515] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:02.349342 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9494c3b6a99487596a030856b371b68" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.349471 1569 raft_consensus.cc:493] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.349577 1569 raft_consensus.cc:515] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.349808 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9494c3b6a99487596a030856b371b68" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:02.349951 854 leader_election.cc:343] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.350291 1569 leader_election.cc:290] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:02.350322 1693 leader_election.cc:290] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:02.350592 856 leader_election.cc:343] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.350731 856 leader_election.cc:304] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.351079 1693 raft_consensus.cc:2749] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.351558 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d1ce00e487d4702a6affcad57a7fbe1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.351929 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d1ce00e487d4702a6affcad57a7fbe1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.352056 856 leader_election.cc:343] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.352402 854 leader_election.cc:343] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.352560 854 leader_election.cc:304] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.353001 1693 raft_consensus.cc:2749] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.353204 1176 tablet_bootstrap.cc:492] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.353380 1176 ts_tablet_manager.cc:1403] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.105s user 0.002s sys 0.005s
I20260505 07:54:02.353826 1181 tablet_bootstrap.cc:492] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.354039 1181 ts_tablet_manager.cc:1403] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.041s user 0.001s sys 0.005s
I20260505 07:54:02.354809 1172 tablet_bootstrap.cc:492] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.354969 1172 ts_tablet_manager.cc:1403] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.014s
I20260505 07:54:02.354830 1181 raft_consensus.cc:359] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.355191 1181 raft_consensus.cc:385] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.355299 1181 raft_consensus.cc:740] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.355613 1181 consensus_queue.cc:260] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.355911 1172 raft_consensus.cc:359] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.356102 1172 raft_consensus.cc:385] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.356195 1172 raft_consensus.cc:740] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.356415 1172 consensus_queue.cc:260] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.356911 1172 ts_tablet_manager.cc:1434] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:02.357231 1176 raft_consensus.cc:359] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.357424 1176 raft_consensus.cc:385] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.357547 1176 raft_consensus.cc:740] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.356560 1181 ts_tablet_manager.cc:1434] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:02.358000 1181 tablet_bootstrap.cc:492] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.357337 1722 tablet.cc:1724] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:02.360345 1181 tablet_bootstrap.cc:654] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.357784 1176 consensus_queue.cc:260] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.362280 1176 ts_tablet_manager.cc:1434] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
W20260505 07:54:02.364734 990 connection.cc:570] server connection from 127.0.0.1:45248 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260505 07:54:02.365516 1317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.105.60:35401 (error 108)
I20260505 07:54:02.366978 1176 tablet_bootstrap.cc:492] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.362910 1172 tablet_bootstrap.cc:492] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.369827 1172 tablet_bootstrap.cc:654] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.372326 1176 tablet_bootstrap.cc:654] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:02.373937 989 connection.cc:570] client connection to 127.0.105.60:35401 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260505 07:54:02.374810 1108 master_proxy_rpc.cc:203] Re-attempting AlterTable request to leader Master (127.0.105.60:35401)
W20260505 07:54:02.375018 716 connection.cc:570] client connection to 127.0.105.60:35401 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260505 07:54:02.375227 1108 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.0.105.60:35401)
W20260505 07:54:02.375306 855 connection.cc:570] client connection to 127.0.105.60:35401 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260505 07:54:02.375654 1108 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.0.105.60:35401)
W20260505 07:54:02.376354 1108 master_proxy_rpc.cc:203] Re-attempting DeleteTable request to leader Master (127.0.105.60:35401)
I20260505 07:54:02.390323 1181 tablet_bootstrap.cc:492] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.390550 1181 ts_tablet_manager.cc:1403] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.001s
I20260505 07:54:02.393095 1181 raft_consensus.cc:359] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.393358 1181 raft_consensus.cc:385] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.393513 1181 raft_consensus.cc:740] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.393832 1181 consensus_queue.cc:260] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.394398 1181 ts_tablet_manager.cc:1434] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:02.396273 1181 tablet_bootstrap.cc:492] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.398301 1181 tablet_bootstrap.cc:654] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.401309 1172 tablet_bootstrap.cc:492] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.401471 1172 ts_tablet_manager.cc:1403] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.010s
I20260505 07:54:02.402215 1172 raft_consensus.cc:359] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.402412 1172 raft_consensus.cc:385] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.402482 1172 raft_consensus.cc:740] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.402694 1172 consensus_queue.cc:260] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.400874 1713 consensus_queue.cc:1048] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.409034 1172 ts_tablet_manager.cc:1434] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:54:02.409337 1172 tablet_bootstrap.cc:492] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.412563 1710 consensus_queue.cc:1048] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.414006 1181 tablet_bootstrap.cc:492] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.414144 1181 ts_tablet_manager.cc:1403] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.002s
I20260505 07:54:02.415191 1172 tablet_bootstrap.cc:654] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.423107 1693 raft_consensus.cc:493] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.423091 1181 raft_consensus.cc:359] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.423305 1181 raft_consensus.cc:385] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.423372 1181 raft_consensus.cc:740] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.423291 1693 raft_consensus.cc:515] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.423629 1181 consensus_queue.cc:260] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.424017 1693 leader_election.cc:290] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:02.425031 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17327cee69a24e0cb493ff55f948bf6c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.425045 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17327cee69a24e0cb493ff55f948bf6c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.425557 854 leader_election.cc:343] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.425693 856 leader_election.cc:343] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.425829 856 leader_election.cc:304] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.426337 1693 raft_consensus.cc:2749] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:02.429040 1317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.60:35401, user_credentials={real_user=slave}} blocked reactor thread for 63514.2us
I20260505 07:54:02.430572 1172 tablet_bootstrap.cc:492] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.430786 1172 ts_tablet_manager.cc:1403] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.003s
I20260505 07:54:02.432874 1172 raft_consensus.cc:359] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:02.433597 1172 raft_consensus.cc:385] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.433883 1172 raft_consensus.cc:740] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.435710 1172 consensus_queue.cc:260] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:02.436627 1172 ts_tablet_manager.cc:1434] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:54:02.437340 1181 ts_tablet_manager.cc:1434] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.023s user 0.000s sys 0.002s
I20260505 07:54:02.437701 1181 tablet_bootstrap.cc:492] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.438868 1172 tablet_bootstrap.cc:492] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.440757 1176 tablet_bootstrap.cc:492] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.440914 1176 ts_tablet_manager.cc:1403] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.078s user 0.001s sys 0.013s
I20260505 07:54:02.443678 1172 tablet_bootstrap.cc:654] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.444803 1176 raft_consensus.cc:359] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.445008 1176 raft_consensus.cc:385] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.445127 1176 raft_consensus.cc:740] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.445365 1176 consensus_queue.cc:260] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.445904 1176 ts_tablet_manager.cc:1434] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.000s sys 0.000s
I20260505 07:54:02.446244 1176 tablet_bootstrap.cc:492] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.447989 1176 tablet_bootstrap.cc:654] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.450258 1181 tablet_bootstrap.cc:654] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:02.452268 1317 leader_election.cc:336] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401): Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
W20260505 07:54:02.452683 1317 leader_election.cc:336] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401): Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:54:02.455073 1176 tablet_bootstrap.cc:492] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.455271 1176 ts_tablet_manager.cc:1403] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260505 07:54:02.456234 1176 raft_consensus.cc:359] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.456441 1176 raft_consensus.cc:385] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.456564 1176 raft_consensus.cc:740] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.456821 1176 consensus_queue.cc:260] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.457576 1176 ts_tablet_manager.cc:1434] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260505 07:54:02.457834 1317 consensus_peers.cc:597] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 -> Peer 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401): Couldn't send request to peer 28d3eda04f9c47ae976d19c218b6dcc5. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:02.457896 1176 tablet_bootstrap.cc:492] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.460620 1176 tablet_bootstrap.cc:654] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.466758 1693 raft_consensus.cc:493] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.466995 1693 raft_consensus.cc:515] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.467984 1693 leader_election.cc:290] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.468911 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e62b7af9b847b7a9005e4345905c0d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.469796 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e62b7af9b847b7a9005e4345905c0d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.470520 854 leader_election.cc:343] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.470990 856 leader_election.cc:343] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.471153 856 leader_election.cc:304] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.472322 1693 raft_consensus.cc:2749] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.475031 1176 tablet_bootstrap.cc:492] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.475222 1176 ts_tablet_manager.cc:1403] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.009s
I20260505 07:54:02.475438 1172 tablet_bootstrap.cc:492] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.475621 1172 ts_tablet_manager.cc:1403] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.000s
I20260505 07:54:02.476131 1176 raft_consensus.cc:359] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.476302 1176 raft_consensus.cc:385] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.476414 1176 raft_consensus.cc:740] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.476399 1172 raft_consensus.cc:359] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.476580 1172 raft_consensus.cc:385] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.476658 1172 raft_consensus.cc:740] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.476706 1176 consensus_queue.cc:260] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.476850 1181 tablet_bootstrap.cc:492] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.476907 1172 consensus_queue.cc:260] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.477368 1172 ts_tablet_manager.cc:1434] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:02.477414 1181 ts_tablet_manager.cc:1403] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.020s
I20260505 07:54:02.477698 1172 tablet_bootstrap.cc:492] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.478511 1181 raft_consensus.cc:359] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.478766 1181 raft_consensus.cc:385] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.478910 1181 raft_consensus.cc:740] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.479159 1181 consensus_queue.cc:260] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.479571 1172 tablet_bootstrap.cc:654] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.479977 1181 ts_tablet_manager.cc:1434] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:02.480441 1181 tablet_bootstrap.cc:492] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.483659 1693 raft_consensus.cc:493] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.483858 1693 raft_consensus.cc:515] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.484637 1693 leader_election.cc:290] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.485329 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "067c32ecccb44bca8bf9cc3f6c229326" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.487882 854 leader_election.cc:343] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.488608 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "067c32ecccb44bca8bf9cc3f6c229326" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:02.489524 856 leader_election.cc:343] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.489717 856 leader_election.cc:304] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.489812 1181 tablet_bootstrap.cc:654] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.490846 1172 tablet_bootstrap.cc:492] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.490988 1172 ts_tablet_manager.cc:1403] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260505 07:54:02.491700 1172 raft_consensus.cc:359] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.491873 1172 raft_consensus.cc:385] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.491956 1172 raft_consensus.cc:740] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.492195 1172 consensus_queue.cc:260] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.492458 1569 raft_consensus.cc:2749] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.495771 1172 ts_tablet_manager.cc:1434] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:02.496966 1693 raft_consensus.cc:493] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.497090 1693 raft_consensus.cc:515] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.497677 1693 leader_election.cc:290] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.498294 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78a6470720984761a8606abeafbef992" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.498905 854 leader_election.cc:343] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.500038 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78a6470720984761a8606abeafbef992" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.500159 1176 ts_tablet_manager.cc:1434] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20260505 07:54:02.500249 1055 raft_consensus.cc:2468] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:02.500496 1176 tablet_bootstrap.cc:492] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.500926 856 leader_election.cc:304] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.502341 1176 tablet_bootstrap.cc:654] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.505975 1172 tablet_bootstrap.cc:492] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:02.506378 718 consensus_peers.cc:597] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9959be9f83c2444282d012b61eef49b5. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:02.507519 1181 tablet_bootstrap.cc:492] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.507772 1172 tablet_bootstrap.cc:654] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.507818 1181 ts_tablet_manager.cc:1403] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.027s user 0.009s sys 0.000s
I20260505 07:54:02.508735 1181 raft_consensus.cc:359] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.509037 1181 raft_consensus.cc:385] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.509159 1181 raft_consensus.cc:740] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.509744 1181 consensus_queue.cc:260] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.510767 1181 ts_tablet_manager.cc:1434] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:02.511373 1181 tablet_bootstrap.cc:492] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.511379 1693 raft_consensus.cc:2804] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.511529 1693 raft_consensus.cc:493] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.511648 1693 raft_consensus.cc:3060] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.513020 1181 tablet_bootstrap.cc:654] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.513850 1176 tablet_bootstrap.cc:492] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.514003 1176 ts_tablet_manager.cc:1403] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260505 07:54:02.514616 1693 raft_consensus.cc:515] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.514791 1176 raft_consensus.cc:359] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.514967 1176 raft_consensus.cc:385] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.515041 1176 raft_consensus.cc:740] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.515359 1176 consensus_queue.cc:260] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.515904 1176 ts_tablet_manager.cc:1434] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:02.516209 1176 tablet_bootstrap.cc:492] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.517360 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78a6470720984761a8606abeafbef992" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:02.517623 1055 raft_consensus.cc:3060] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.517930 1176 tablet_bootstrap.cc:654] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.521117 1569 raft_consensus.cc:493] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.521306 1569 raft_consensus.cc:515] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.521994 1569 leader_election.cc:290] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.524439 1569 raft_consensus.cc:493] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.524658 1569 raft_consensus.cc:515] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.524758 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4783dd0b2fbe40219852d850fc0d9a82" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.525290 1569 leader_election.cc:290] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:02.525501 856 leader_election.cc:343] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.526052 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.526815 1055 raft_consensus.cc:2468] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
W20260505 07:54:02.529085 856 leader_election.cc:343] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.529395 856 leader_election.cc:304] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:02.529860 1569 raft_consensus.cc:2804] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.534679 1181 tablet_bootstrap.cc:492] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.534884 1181 ts_tablet_manager.cc:1403] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.002s
I20260505 07:54:02.537813 1181 raft_consensus.cc:359] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.538031 1181 raft_consensus.cc:385] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.538137 1181 raft_consensus.cc:740] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.538435 1181 consensus_queue.cc:260] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.540298 1181 ts_tablet_manager.cc:1434] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:54:02.541627 1181 tablet_bootstrap.cc:492] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.544116 1181 tablet_bootstrap.cc:654] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.551172 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4783dd0b2fbe40219852d850fc0d9a82" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.551260 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.551749 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78a6470720984761a8606abeafbef992" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:02.552498 854 leader_election.cc:343] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.552687 854 leader_election.cc:304] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.554108 1181 tablet_bootstrap.cc:492] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.554309 1181 ts_tablet_manager.cc:1403] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260505 07:54:02.555326 1181 raft_consensus.cc:359] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.555552 1181 raft_consensus.cc:385] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.555661 1181 raft_consensus.cc:740] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.555971 1181 consensus_queue.cc:260] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.556089 1693 leader_election.cc:290] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.557032 1693 raft_consensus.cc:493] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.557260 1693 raft_consensus.cc:515] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.557878 1181 ts_tablet_manager.cc:1434] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260505 07:54:02.558671 1693 leader_election.cc:290] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.559350 1181 tablet_bootstrap.cc:492] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.561010 1181 tablet_bootstrap.cc:654] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.569870 1693 raft_consensus.cc:2749] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.574509 1693 raft_consensus.cc:493] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.574723 1693 raft_consensus.cc:515] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.575467 1693 leader_election.cc:290] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.578162 1181 tablet_bootstrap.cc:492] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.578333 1181 ts_tablet_manager.cc:1403] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.001s
I20260505 07:54:02.579233 1181 raft_consensus.cc:359] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.579444 1181 raft_consensus.cc:385] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.579543 1181 raft_consensus.cc:740] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.579821 1181 consensus_queue.cc:260] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.580957 1649 raft_consensus.cc:493] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.581141 1649 raft_consensus.cc:515] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.583801 1181 ts_tablet_manager.cc:1434] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260505 07:54:02.585050 1649 leader_election.cc:290] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.585915 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd581a6799a14a8dafb80d411e8936f3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.586094 1055 raft_consensus.cc:2468] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:02.586591 717 leader_election.cc:304] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.586952 1649 raft_consensus.cc:2804] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.587076 1649 raft_consensus.cc:493] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.587177 1649 raft_consensus.cc:3060] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.587363 1181 tablet_bootstrap.cc:492] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.589151 1181 tablet_bootstrap.cc:654] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.592983 1649 raft_consensus.cc:515] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.594383 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bd581a6799a14a8dafb80d411e8936f3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:02.594599 1055 raft_consensus.cc:3060] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.597003 1569 raft_consensus.cc:697] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:02.597417 1569 consensus_queue.cc:237] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.598351 1055 raft_consensus.cc:2468] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:02.598856 854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 46415.3us
W20260505 07:54:02.599253 854 leader_election.cc:343] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.599380 854 leader_election.cc:304] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:02.599905 854 leader_election.cc:343] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.600631 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70404184502a4394a737518220f2124f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.600677 717 leader_election.cc:304] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.601068 1512 raft_consensus.cc:2804] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.601233 1512 raft_consensus.cc:697] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:02.601598 1512 consensus_queue.cc:237] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:02.602247 718 leader_election.cc:343] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: bd581a6799a14a8dafb80d411e8936f3
W20260505 07:54:02.602684 854 leader_election.cc:343] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.603185 1749 raft_consensus.cc:2749] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.603395 1749 raft_consensus.cc:493] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.603595 1749 raft_consensus.cc:515] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.604005 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003cd4472b424aacbba8c5f89900e228" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.604511 1749 leader_election.cc:290] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:02.604897 856 leader_election.cc:343] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.605254 1748 raft_consensus.cc:493] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.605417 1748 raft_consensus.cc:515] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:02.605881 991 consensus_peers.cc:597] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ccdde3e5a13e436d9f2328831e155adb. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:02.606130 1748 leader_election.cc:290] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.606197 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003cd4472b424aacbba8c5f89900e228" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.607187 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a06f77255544d7dabb71a4f70ada809" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.607425 1053 raft_consensus.cc:2468] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:02.607770 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac5064f68d24ebe95b5a0b8e10f463e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.607975 1052 raft_consensus.cc:2468] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
W20260505 07:54:02.608131 854 leader_election.cc:343] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.608779 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a06f77255544d7dabb71a4f70ada809" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.608883 854 leader_election.cc:304] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.609112 1748 raft_consensus.cc:2749] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:02.609268 854 leader_election.cc:343] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.609398 1172 tablet_bootstrap.cc:492] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.609388 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac5064f68d24ebe95b5a0b8e10f463e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.609553 1172 ts_tablet_manager.cc:1403] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.104s user 0.012s sys 0.038s
I20260505 07:54:02.610121 856 leader_election.cc:304] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.610294 1172 raft_consensus.cc:359] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.610463 1172 raft_consensus.cc:385] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.610538 1172 raft_consensus.cc:740] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.610710 856 leader_election.cc:304] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:02.610828 1172 consensus_queue.cc:260] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.611124 1751 raft_consensus.cc:2804] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.611339 1172 ts_tablet_manager.cc:1434] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:02.611271 1751 raft_consensus.cc:493] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.611832 1751 raft_consensus.cc:3060] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.611937 1172 tablet_bootstrap.cc:492] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.611128 1749 raft_consensus.cc:2804] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.612423 1749 raft_consensus.cc:493] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.612591 1749 raft_consensus.cc:3060] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.614564 1176 tablet_bootstrap.cc:492] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.614717 1176 ts_tablet_manager.cc:1403] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.099s user 0.011s sys 0.035s
I20260505 07:54:02.615453 1176 raft_consensus.cc:359] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.615639 1176 raft_consensus.cc:385] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.615760 1176 raft_consensus.cc:740] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.613816 1172 tablet_bootstrap.cc:654] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.616884 1176 consensus_queue.cc:260] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.617714 1751 raft_consensus.cc:515] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.618160 1176 ts_tablet_manager.cc:1434] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260505 07:54:02.619335 1176 tablet_bootstrap.cc:492] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.620214 1751 leader_election.cc:290] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.619297 1749 raft_consensus.cc:515] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.621351 1176 tablet_bootstrap.cc:654] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.621305 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac5064f68d24ebe95b5a0b8e10f463e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:02.621515 1052 raft_consensus.cc:3060] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.621697 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a06f77255544d7dabb71a4f70ada809" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:02.621933 1053 raft_consensus.cc:3060] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.625372 1052 raft_consensus.cc:2468] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:02.625545 1053 raft_consensus.cc:2468] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:02.625883 856 leader_election.cc:304] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:02.626365 856 leader_election.cc:304] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:02.626644 1748 raft_consensus.cc:2804] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.626749 1751 raft_consensus.cc:2804] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.626794 1748 raft_consensus.cc:697] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:02.627102 1748 consensus_queue.cc:237] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.627669 1749 leader_election.cc:290] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.628489 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70404184502a4394a737518220f2124f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:02.629038 856 leader_election.cc:343] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.629156 856 leader_election.cc:304] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.629874 1758 raft_consensus.cc:2749] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.626519 1176 tablet_bootstrap.cc:492] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.632158 1176 ts_tablet_manager.cc:1403] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.003s
I20260505 07:54:02.630882 1751 raft_consensus.cc:697] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:02.632827 1751 consensus_queue.cc:237] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.633316 1176 raft_consensus.cc:359] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.633491 1176 raft_consensus.cc:385] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.633606 1176 raft_consensus.cc:740] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.630254 1649 leader_election.cc:290] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.635403 1172 tablet_bootstrap.cc:492] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.635524 1172 ts_tablet_manager.cc:1403] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.003s
I20260505 07:54:02.636170 1172 raft_consensus.cc:359] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.636322 1172 raft_consensus.cc:385] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.636402 1172 raft_consensus.cc:740] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.636603 1172 consensus_queue.cc:260] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.637248 1172 ts_tablet_manager.cc:1434] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:02.636909 1649 raft_consensus.cc:493] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.637555 1172 tablet_bootstrap.cc:492] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.637562 1649 raft_consensus.cc:515] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.639474 1172 tablet_bootstrap.cc:654] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.639685 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585f00119e14006990ed551667b4486" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.639926 1055 raft_consensus.cc:2468] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:02.640340 717 leader_election.cc:304] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.640575 1176 consensus_queue.cc:260] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.640834 1512 raft_consensus.cc:2804] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.640944 1512 raft_consensus.cc:493] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.641031 1512 raft_consensus.cc:3060] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.641153 1176 ts_tablet_manager.cc:1434] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260505 07:54:02.641541 1176 tablet_bootstrap.cc:492] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.641963 1649 leader_election.cc:290] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.643069 1176 tablet_bootstrap.cc:654] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.646034 1172 tablet_bootstrap.cc:492] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.646219 1172 ts_tablet_manager.cc:1403] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20260505 07:54:02.647156 1172 raft_consensus.cc:359] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.647228 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a06f77255544d7dabb71a4f70ada809" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.647389 1172 raft_consensus.cc:385] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.647522 1172 raft_consensus.cc:740] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.647819 1172 consensus_queue.cc:260] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.648236 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac5064f68d24ebe95b5a0b8e10f463e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.648511 1172 ts_tablet_manager.cc:1434] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260505 07:54:02.648808 1172 tablet_bootstrap.cc:492] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:02.649923 854 leader_election.cc:343] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.651014 854 leader_election.cc:343] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.651399 854 leader_election.cc:343] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.652110 718 leader_election.cc:343] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: bd581a6799a14a8dafb80d411e8936f3
W20260505 07:54:02.652585 718 leader_election.cc:343] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 7585f00119e14006990ed551667b4486
I20260505 07:54:02.658635 467 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Refusing update from remote peer 23c57a2c927c4c54bccecfeb3a775311: Log matching property violated. Preceding OpId in replica: term: 1 index: 589. Preceding OpId from leader: term: 2 index: 591. (index mismatch)
I20260505 07:54:02.660000 1679 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [LEADER]: Connected to new peer: Peer: permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 591, Last known committed idx: 589, Time since last communication: 0.000s
I20260505 07:54:02.660730 1569 raft_consensus.cc:493] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.660890 1569 raft_consensus.cc:515] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.662015 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397d33e908af4a0bb8dde26d273be01a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.662530 1512 raft_consensus.cc:515] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:02.662922 856 leader_election.cc:343] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.663478 1181 tablet_bootstrap.cc:492] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.663661 1181 ts_tablet_manager.cc:1403] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.076s user 0.026s sys 0.020s
I20260505 07:54:02.663684 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7585f00119e14006990ed551667b4486" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:02.663844 1055 raft_consensus.cc:3060] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.664117 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397d33e908af4a0bb8dde26d273be01a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.664476 1181 raft_consensus.cc:359] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.664690 1181 raft_consensus.cc:385] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260505 07:54:02.664745 854 leader_election.cc:343] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.664783 1181 raft_consensus.cc:740] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.664875 854 leader_election.cc:304] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.665166 1758 raft_consensus.cc:2749] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.665378 1569 leader_election.cc:290] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.665786 1172 tablet_bootstrap.cc:654] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.667013 1055 raft_consensus.cc:2468] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:02.667400 717 leader_election.cc:304] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.667749 1649 raft_consensus.cc:2804] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.667910 1649 raft_consensus.cc:697] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:02.668169 1649 consensus_queue.cc:237] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.670656 1176 tablet_bootstrap.cc:492] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.671096 1176 ts_tablet_manager.cc:1403] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.030s user 0.008s sys 0.011s
I20260505 07:54:02.672009 1176 raft_consensus.cc:359] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.675040 1176 raft_consensus.cc:385] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.676256 1176 raft_consensus.cc:740] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.676438 1181 consensus_queue.cc:260] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.676955 1176 consensus_queue.cc:260] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.679234 1176 ts_tablet_manager.cc:1434] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20260505 07:54:02.680294 1176 tablet_bootstrap.cc:492] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.682750 1176 tablet_bootstrap.cc:654] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.683513 1751 raft_consensus.cc:493] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.683673 1751 raft_consensus.cc:515] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:02.685578 1751 leader_election.cc:290] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:02.686486 1172 tablet_bootstrap.cc:492] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.686622 1172 ts_tablet_manager.cc:1403] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.038s user 0.002s sys 0.005s
I20260505 07:54:02.687270 1172 raft_consensus.cc:359] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.687418 1172 raft_consensus.cc:385] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.687525 1172 raft_consensus.cc:740] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.687762 1172 consensus_queue.cc:260] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.688336 1172 ts_tablet_manager.cc:1434] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:02.688956 1172 tablet_bootstrap.cc:492] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.690551 1172 tablet_bootstrap.cc:654] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.691469 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c822fafe72544ca99fde910c8e764755" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:02.693553 856 leader_election.cc:343] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.685403 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c822fafe72544ca99fde910c8e764755" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.698628 854 leader_election.cc:343] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.698760 854 leader_election.cc:304] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.699123 1751 raft_consensus.cc:2749] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.696029 1512 leader_election.cc:290] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:02.696798 718 leader_election.cc:343] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 7585f00119e14006990ed551667b4486
I20260505 07:54:02.697633 1649 consensus_queue.cc:1048] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.702251 1751 raft_consensus.cc:493] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.702375 1751 raft_consensus.cc:515] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.702618 1181 ts_tablet_manager.cc:1434] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.039s user 0.002s sys 0.000s
I20260505 07:54:02.702922 1751 leader_election.cc:290] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.703209 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141665fcd59048bf926acd0287f1853e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.703457 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141665fcd59048bf926acd0287f1853e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.703737 856 leader_election.cc:343] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.703943 854 leader_election.cc:343] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.704042 854 leader_election.cc:304] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.704115 1181 tablet_bootstrap.cc:492] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.704418 1751 raft_consensus.cc:2749] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.705965 1181 tablet_bootstrap.cc:654] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.706096 1512 raft_consensus.cc:493] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.706245 1512 raft_consensus.cc:515] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.707697 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d485ae2d1d4d76a6a5d988cec8291a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.708103 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d485ae2d1d4d76a6a5d988cec8291a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:02.708289 921 raft_consensus.cc:2468] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:02.708738 718 leader_election.cc:304] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.708990 1649 raft_consensus.cc:2804] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.709075 1649 raft_consensus.cc:493] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.709137 1649 raft_consensus.cc:3060] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.711699 1649 raft_consensus.cc:515] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.712551 1649 leader_election.cc:290] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.713016 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d485ae2d1d4d76a6a5d988cec8291a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:02.713184 921 raft_consensus.cc:3060] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:02.714116 717 leader_election.cc:343] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.714164 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d485ae2d1d4d76a6a5d988cec8291a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:02.716528 921 raft_consensus.cc:2468] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:02.716849 717 leader_election.cc:343] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.717320 718 leader_election.cc:304] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:02.717744 1649 raft_consensus.cc:2804] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.718113 1649 raft_consensus.cc:697] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:02.718425 1649 consensus_queue.cc:237] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.719674 1512 leader_election.cc:290] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.721900 1181 tablet_bootstrap.cc:492] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.722041 1181 ts_tablet_manager.cc:1403] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.001s
I20260505 07:54:02.722486 1668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23c57a2c927c4c54bccecfeb3a775311. Latest consensus state: current_term: 2 leader_uuid: "23c57a2c927c4c54bccecfeb3a775311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:54:02.722676 1668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:54:02.722901 1181 raft_consensus.cc:359] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.723091 1181 raft_consensus.cc:385] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.723177 1181 raft_consensus.cc:740] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.723397 1181 consensus_queue.cc:260] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.726344 1172 tablet_bootstrap.cc:492] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.726498 1172 ts_tablet_manager.cc:1403] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.038s user 0.006s sys 0.000s
I20260505 07:54:02.727242 1172 raft_consensus.cc:359] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.727403 1172 raft_consensus.cc:385] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.727468 1172 raft_consensus.cc:740] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.729573 1172 consensus_queue.cc:260] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.731199 1172 ts_tablet_manager.cc:1434] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:02.732273 1172 tablet_bootstrap.cc:492] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.734700 1172 tablet_bootstrap.cc:654] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.744498 1181 ts_tablet_manager.cc:1434] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
W20260505 07:54:02.746675 991 consensus_peers.cc:597] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8a60c015136246a3b02f70254c0bb67f. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:02.749094 1181 tablet_bootstrap.cc:492] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.750566 1181 tablet_bootstrap.cc:654] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.752377 1181 tablet_bootstrap.cc:492] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.752518 1181 ts_tablet_manager.cc:1403] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20260505 07:54:02.753177 1181 raft_consensus.cc:359] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.753350 1181 raft_consensus.cc:385] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.753414 1181 raft_consensus.cc:740] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.753639 1181 consensus_queue.cc:260] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.754472 1751 raft_consensus.cc:493] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.754665 1751 raft_consensus.cc:515] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.754818 1181 ts_tablet_manager.cc:1434] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:02.755386 1751 leader_election.cc:290] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.755968 1181 tablet_bootstrap.cc:492] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.755999 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02412333633a4cdebe95a144fa6c363c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:02.756532 856 leader_election.cc:343] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.757130 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02412333633a4cdebe95a144fa6c363c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.757723 1181 tablet_bootstrap.cc:654] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:02.757710 854 leader_election.cc:343] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.757838 854 leader_election.cc:304] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.758365 1751 raft_consensus.cc:2749] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.759582 1181 tablet_bootstrap.cc:492] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.759727 1181 ts_tablet_manager.cc:1403] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260505 07:54:02.760382 1181 raft_consensus.cc:359] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.761209 1176 tablet_bootstrap.cc:492] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.761364 1176 ts_tablet_manager.cc:1403] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.081s user 0.002s sys 0.007s
W20260505 07:54:02.761946 991 consensus_peers.cc:597] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1852f7f4ff594639908b03730b68fe86. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:02.760540 1181 raft_consensus.cc:385] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.762048 1176 raft_consensus.cc:359] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.762238 1176 raft_consensus.cc:385] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.762169 1181 raft_consensus.cc:740] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.762307 1176 raft_consensus.cc:740] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.763082 1181 consensus_queue.cc:260] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.763952 1181 ts_tablet_manager.cc:1434] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260505 07:54:02.764211 1181 tablet_bootstrap.cc:492] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.765362 1181 tablet_bootstrap.cc:654] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.766821 1181 tablet_bootstrap.cc:492] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.766979 1181 ts_tablet_manager.cc:1403] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260505 07:54:02.767634 1181 raft_consensus.cc:359] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.768121 1181 raft_consensus.cc:385] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.768210 1181 raft_consensus.cc:740] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.768435 1181 consensus_queue.cc:260] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.768958 1181 ts_tablet_manager.cc:1434] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:02.769238 1181 tablet_bootstrap.cc:492] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.770781 1181 tablet_bootstrap.cc:654] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.772387 1181 tablet_bootstrap.cc:492] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:02.772367 1668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "23c57a2c927c4c54bccecfeb3a775311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:54:02.772584 1668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:54:02.772687 1181 ts_tablet_manager.cc:1403] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260505 07:54:02.772965 1176 consensus_queue.cc:260] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.773458 1176 ts_tablet_manager.cc:1434] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.012s user 0.000s sys 0.001s
I20260505 07:54:02.773573 1181 raft_consensus.cc:359] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.773880 1181 raft_consensus.cc:385] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.774024 1181 raft_consensus.cc:740] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.774271 1181 consensus_queue.cc:260] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.774765 1172 tablet_bootstrap.cc:492] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:02.774914 1172 ts_tablet_manager.cc:1403] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.043s user 0.003s sys 0.005s
I20260505 07:54:02.775076 1181 ts_tablet_manager.cc:1434] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:02.775695 1181 tablet_bootstrap.cc:492] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:02.775729 1172 raft_consensus.cc:359] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.775902 1172 raft_consensus.cc:385] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.775990 1172 raft_consensus.cc:740] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.776209 1172 consensus_queue.cc:260] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.776891 1779 raft_consensus.cc:493] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.777040 1779 raft_consensus.cc:515] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.778119 1779 leader_election.cc:290] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.779009 1181 tablet_bootstrap.cc:654] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.781872 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92c53c68a4b4e319e98b84128e8e6a4" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.782063 1055 raft_consensus.cc:2468] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
W20260505 07:54:02.785984 718 leader_election.cc:343] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: f92c53c68a4b4e319e98b84128e8e6a4
I20260505 07:54:02.786552 717 leader_election.cc:304] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:02.787022 1649 raft_consensus.cc:2804] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.787137 1649 raft_consensus.cc:493] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.787225 1649 raft_consensus.cc:3060] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.785969 1176 tablet_bootstrap.cc:492] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.791934 1567 consensus_queue.cc:1048] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.793040 1176 tablet_bootstrap.cc:654] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.796645 1649 raft_consensus.cc:515] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.797830 1176 tablet_bootstrap.cc:492] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.797995 1176 ts_tablet_manager.cc:1403] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.000s
I20260505 07:54:02.798750 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f92c53c68a4b4e319e98b84128e8e6a4" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:02.798942 1055 raft_consensus.cc:3060] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.798915 1176 raft_consensus.cc:359] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.799068 1176 raft_consensus.cc:385] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.799156 1176 raft_consensus.cc:740] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.800310 1679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "23c57a2c927c4c54bccecfeb3a775311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:54:02.800524 1679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: This master's current role is: LEADER
I20260505 07:54:02.802044 1055 raft_consensus.cc:2468] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:02.802871 1176 consensus_queue.cc:260] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.803385 1176 ts_tablet_manager.cc:1434] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:02.803704 1176 tablet_bootstrap.cc:492] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.806370 1176 tablet_bootstrap.cc:654] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.804148 1567 raft_consensus.cc:493] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.810742 1567 raft_consensus.cc:515] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.812286 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1058fcb9fa6457a85489049ca1d9ada" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:02.812543 921 raft_consensus.cc:2468] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.813055 991 leader_election.cc:304] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:02.813378 1596 raft_consensus.cc:2804] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.813510 1596 raft_consensus.cc:493] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.813611 1596 raft_consensus.cc:3060] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.816370 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1058fcb9fa6457a85489049ca1d9ada" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.816636 781 raft_consensus.cc:2468] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.817217 1176 tablet_bootstrap.cc:492] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:02.817371 1176 ts_tablet_manager.cc:1403] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.002s
I20260505 07:54:02.818101 1176 raft_consensus.cc:359] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.818279 1176 raft_consensus.cc:385] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:02.818342 1176 raft_consensus.cc:740] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:02.818593 1176 consensus_queue.cc:260] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.818969 1751 raft_consensus.cc:493] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.819118 1751 raft_consensus.cc:515] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.819780 1751 leader_election.cc:290] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.819840 1710 raft_consensus.cc:493] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.819983 1710 raft_consensus.cc:515] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.820574 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4662d4a3d88d4c2883a35048d6e11eb9" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.820624 1710 leader_election.cc:290] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.820779 781 raft_consensus.cc:2468] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:02.821172 717 leader_election.cc:304] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.821189 854 leader_election.cc:304] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.821488 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca2852cee84476dbe5e5072a43087a1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:02.821668 921 raft_consensus.cc:2468] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.822034 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca2852cee84476dbe5e5072a43087a1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.822132 991 leader_election.cc:304] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:02.822232 781 raft_consensus.cc:2468] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:02.822465 1751 raft_consensus.cc:2804] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.822494 1710 raft_consensus.cc:2804] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.822598 1710 raft_consensus.cc:493] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.822595 1751 raft_consensus.cc:493] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.822714 1751 raft_consensus.cc:3060] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.822747 1710 raft_consensus.cc:3060] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.825506 1751 raft_consensus.cc:515] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.825512 1710 raft_consensus.cc:515] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.827443 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca2852cee84476dbe5e5072a43087a1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:02.827637 921 raft_consensus.cc:3060] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.828977 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ca2852cee84476dbe5e5072a43087a1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.829145 781 raft_consensus.cc:3060] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.830204 921 raft_consensus.cc:2468] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.830688 991 leader_election.cc:304] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:02.830971 1580 raft_consensus.cc:2804] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.831115 1580 raft_consensus.cc:697] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:02.831350 1580 consensus_queue.cc:237] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.831970 781 raft_consensus.cc:2468] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.811517 1567 leader_election.cc:290] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.835736 1596 raft_consensus.cc:515] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.836344 1596 leader_election.cc:290] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.836901 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1058fcb9fa6457a85489049ca1d9ada" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.837087 782 raft_consensus.cc:3060] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.840137 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1058fcb9fa6457a85489049ca1d9ada" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:02.840323 919 raft_consensus.cc:3060] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.840623 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4662d4a3d88d4c2883a35048d6e11eb9" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:02.840798 780 raft_consensus.cc:3060] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.842979 919 raft_consensus.cc:2468] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.843405 780 raft_consensus.cc:2468] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:02.843703 991 leader_election.cc:304] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:02.843955 854 leader_election.cc:304] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:02.844228 1569 raft_consensus.cc:2804] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.844338 1569 raft_consensus.cc:697] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:02.844625 1569 consensus_queue.cc:237] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.845983 782 raft_consensus.cc:2468] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:02.847110 1751 leader_election.cc:290] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.849001 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4662d4a3d88d4c2883a35048d6e11eb9" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:02.849761 856 leader_election.cc:343] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.850786 1176 ts_tablet_manager.cc:1434] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.033s user 0.001s sys 0.001s
I20260505 07:54:02.850804 1512 raft_consensus.cc:2804] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.851150 1176 tablet_bootstrap.cc:492] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:02.851208 1512 raft_consensus.cc:697] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:02.851486 1512 consensus_queue.cc:237] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.851830 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4662d4a3d88d4c2883a35048d6e11eb9" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:02.855140 856 leader_election.cc:343] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.856346 718 consensus_peers.cc:597] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 03f31efbc98c484c950c70d5e84d2136. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:02.857270 1596 raft_consensus.cc:2804] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.857547 1596 raft_consensus.cc:697] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:02.857980 1596 consensus_queue.cc:237] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.860441 1176 tablet_bootstrap.cc:654] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:02.861125 1710 leader_election.cc:290] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.864475 1580 consensus_queue.cc:1048] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.866938 1684 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20260505 07:54:02.874792 1649 leader_election.cc:290] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:02.875666 718 leader_election.cc:343] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: f92c53c68a4b4e319e98b84128e8e6a4
I20260505 07:54:02.878595 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_57b735b31fec4ba88439afddef823171 [id=037074511cd8426487cfdb5298d1ee8e]
I20260505 07:54:02.879848 1710 raft_consensus.cc:493] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.880016 1710 raft_consensus.cc:515] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.880121 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]
I20260505 07:54:02.880769 1710 leader_election.cc:290] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.881104 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d485ae2d1d4d76a6a5d988cec8291a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:02.881289 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d485ae2d1d4d76a6a5d988cec8291a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.881387 921 raft_consensus.cc:2393] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:02.881800 988 leader_election.cc:304] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.882153 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_7b4336be586a4fc1a8dada3651641fd9 [id=057b807a46cb43b4a60dc61212a8b8a8]
I20260505 07:54:02.882263 1710 raft_consensus.cc:3060] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.883101 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]
I20260505 07:54:02.883529 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_6c975d29ebde41ecb90fdaf8c72b2997 [id=0a01d809c30041a0a5b803341584f3ac]
I20260505 07:54:02.884918 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b]
I20260505 07:54:02.886840 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_a4147e267b5f45e78812bde816ae7c8d [id=0b6c6c33d8eb41dd9b351e0d4f0e8bcd]
I20260505 07:54:02.888640 1710 raft_consensus.cc:2749] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.889392 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_4ce294c5bd3f49699d1e194c53ff9c07 [id=0ca5072e233148099ba64cca861c144c]
I20260505 07:54:02.892666 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_db54454473d84a83b6c77d06a1f09618 [id=0faa6676bab04919984b580299143271]
I20260505 07:54:02.893761 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_119772d89d5f4afa8108d78252d81fe6 [id=129e84f069a74fe9b8ee67eaeba930a3]
I20260505 07:54:02.898525 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_a44f4d613a6e47e38e9741252ef6956a [id=1a0d30f064af4c2f8e082d5742ba6d28]
I20260505 07:54:02.901628 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_af7b8d3a4e6843348198094736404edb [id=1a1c2d94983e409c8b75400de8adbae8]
I20260505 07:54:02.901726 1580 consensus_queue.cc:1048] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.903378 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_194f3e5aa086410eb3bd64b872d2f8af [id=1a7d610787c44274a6036482200780cf]
I20260505 07:54:02.907543 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_93d07f8983d94584a9c9700054d460e6 [id=1c2e08372ceb44e7945a4abb176da05e]
I20260505 07:54:02.910717 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]
I20260505 07:54:02.911701 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d [id=1fcae33a3a2d48e7b2000cc7b906c7f5]
I20260505 07:54:02.913153 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_3621941a687641ecbcaa200aaefbe112 [id=2164fb49c5204df0ac8c95667d342af5]
I20260505 07:54:02.913834 1580 consensus_queue.cc:1048] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260505 07:54:02.915571 1580 consensus_queue.cc:1048] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.928354 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_911918ab5f5145178d2490fb80e7db21 [id=218966c8e4af4210a14adcb68269cda4]
I20260505 07:54:02.930413 1649 raft_consensus.cc:493] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.930670 1649 raft_consensus.cc:515] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.931316 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_cf3b3e8080ba449a9f92b5175e2193b1 [id=22feb370028a4bd59568731eaa7b090c]
I20260505 07:54:02.932067 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf76e868bbcb44b0b1887c84dd27f093" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.933245 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_cc284917d2aa41d9956affce2c946825 [id=231630aa148e43d1839edb43f686ad20]
W20260505 07:54:02.933331 717 leader_election.cc:343] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.934065 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]
I20260505 07:54:02.934564 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_a11d31ae6a634b5693db8de72ca33cf8 [id=256562aafb76429fb120899d8af6704d]
W20260505 07:54:02.934733 718 leader_election.cc:343] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: bf76e868bbcb44b0b1887c84dd27f093
I20260505 07:54:02.934895 718 leader_election.cc:304] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:02.935060 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_66f28ba637f24448816a64c7d316edee [id=266ff796c7a74c23816dd9c422e9d005]
I20260505 07:54:02.935195 1512 raft_consensus.cc:2749] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.935272 1798 raft_consensus.cc:493] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:02.935607 718 consensus_peers.cc:597] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a79578757824bfc9df129e3475e7af3. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:02.935559 1798 raft_consensus.cc:515] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.935910 1649 leader_election.cc:290] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.936343 1580 consensus_queue.cc:1048] T fcd36b5a84b4442fb76e1610d8af4c07 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.937119 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d878ae0ddb42319560f74e0a490321" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.937242 1172 ts_tablet_manager.cc:1434] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.157s user 0.002s sys 0.000s
I20260505 07:54:02.937404 780 raft_consensus.cc:2393] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:02.937685 1172 tablet_bootstrap.cc:492] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:02.942236 1710 raft_consensus.cc:493] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.942394 1710 raft_consensus.cc:515] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.943521 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397d33e908af4a0bb8dde26d273be01a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:02.943665 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397d33e908af4a0bb8dde26d273be01a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:02.943859 919 raft_consensus.cc:2468] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:02.944259 988 leader_election.cc:343] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.944514 988 leader_election.cc:304] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.944869 1580 raft_consensus.cc:2804] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.945103 1580 raft_consensus.cc:493] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.945293 1580 raft_consensus.cc:3060] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.945526 1649 raft_consensus.cc:493] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.945762 1649 raft_consensus.cc:515] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.946415 1649 leader_election.cc:290] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.946556 1569 raft_consensus.cc:493] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:02.946733 1569 raft_consensus.cc:515] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.947449 1569 leader_election.cc:290] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:02.947695 1798 leader_election.cc:290] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.947795 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93d878ae0ddb42319560f74e0a490321" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:02.947964 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06904a54b1c5463da31af0bc00fb606f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:02.948590 854 leader_election.cc:343] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.948978 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa401bb4634022bca3c6b23421e4d5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.949177 991 leader_election.cc:304] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:02.949910 717 leader_election.cc:343] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.950280 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa401bb4634022bca3c6b23421e4d5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:02.950534 919 raft_consensus.cc:2468] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:02.951164 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06904a54b1c5463da31af0bc00fb606f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:02.951633 718 leader_election.cc:304] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
W20260505 07:54:02.951721 856 leader_election.cc:343] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:02.951877 856 leader_election.cc:304] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:02.952282 1649 raft_consensus.cc:2804] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:02.952315 1569 raft_consensus.cc:2749] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.952436 1649 raft_consensus.cc:493] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:02.952553 1649 raft_consensus.cc:3060] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.952605 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_c1551c1a06c34c5dae05b3d5f1109d43 [id=276ccf1ef8a9474dbb936d880f10805e]
I20260505 07:54:02.954730 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958]
I20260505 07:54:02.955293 1801 raft_consensus.cc:3060] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.955626 1710 leader_election.cc:290] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.957137 1710 consensus_queue.cc:1048] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:02.960108 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_a930977d21a04e229868a60d643ca173 [id=28164b2107de472ab15c913fbbc9840b]
I20260505 07:54:02.962622 1801 raft_consensus.cc:2749] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:02.963650 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_ed8e2add9bbf436f812de7e3fe2ed692 [id=2a512c2695b540c2a9e14616ee257125]
I20260505 07:54:02.965566 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]
I20260505 07:54:02.958989 1649 raft_consensus.cc:515] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.968214 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_268b9129e7fb4560bcbbb7e5427975fa [id=2c06ff8eed31432e87f08e9b59049652]
I20260505 07:54:02.968932 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa401bb4634022bca3c6b23421e4d5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:02.969506 717 leader_election.cc:343] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:02.970348 991 consensus_peers.cc:597] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 909306e1e5f34b7c8fbc0d86cafce531. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:02.971020 1649 leader_election.cc:290] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.972685 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_829cc346ac834637ab2e2198944a8550 [id=2c91868ddba84103b9a5c78201842a9e]
I20260505 07:54:02.973374 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa401bb4634022bca3c6b23421e4d5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:02.973553 919 raft_consensus.cc:3060] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.976532 919 raft_consensus.cc:2468] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:02.977315 718 leader_election.cc:304] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:02.977908 1580 raft_consensus.cc:515] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.978186 1649 raft_consensus.cc:2804] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:02.978295 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_9201abe14fd54f8880602582b1f74cc8 [id=2f4387e6af20481fa79a3b8849a529cf]
I20260505 07:54:02.979528 1649 raft_consensus.cc:697] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:02.979965 1649 consensus_queue.cc:237] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:02.980232 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_7e8b9279f3e244a99b8ba4a25e7c569f [id=306149ae47a745edabf863d5004d5012]
I20260505 07:54:02.981353 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]
I20260505 07:54:02.982758 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_6f171a1e79d24e57acfe3aef43b24c7e [id=35a56d340665450ab03ba49b845ae78a]
I20260505 07:54:02.987833 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397d33e908af4a0bb8dde26d273be01a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:02.988099 919 raft_consensus.cc:3060] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:02.989933 1580 leader_election.cc:290] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:02.997269 919 raft_consensus.cc:2468] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:03.000480 991 leader_election.cc:304] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:03.001010 1580 raft_consensus.cc:2804] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.001067 1512 raft_consensus.cc:493] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.001220 1580 raft_consensus.cc:697] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:03.001226 1512 raft_consensus.cc:515] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.001677 1580 consensus_queue.cc:237] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.001837 1512 leader_election.cc:290] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.002859 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e278b86e7b5d444ebad8d0f6921d4c62" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.002851 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e278b86e7b5d444ebad8d0f6921d4c62" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.003034 1052 raft_consensus.cc:2468] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.003034 919 raft_consensus.cc:2468] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.003429 718 leader_election.cc:304] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.003924 1512 raft_consensus.cc:2804] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.004031 1512 raft_consensus.cc:493] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.004145 1512 raft_consensus.cc:3060] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.007860 1512 raft_consensus.cc:515] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.008451 1645 consensus_queue.cc:1048] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.008914 1512 leader_election.cc:290] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.008960 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e278b86e7b5d444ebad8d0f6921d4c62" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.009346 921 raft_consensus.cc:3060] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.009833 1172 tablet_bootstrap.cc:654] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.010213 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e278b86e7b5d444ebad8d0f6921d4c62" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:03.010432 1052 raft_consensus.cc:3060] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.014462 1052 raft_consensus.cc:2468] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:03.015221 717 leader_election.cc:304] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.015573 1512 raft_consensus.cc:2804] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.015856 1512 raft_consensus.cc:697] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:03.016112 1512 consensus_queue.cc:237] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.019016 921 raft_consensus.cc:2468] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:03.025102 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 62575.1us
W20260505 07:54:03.029359 991 consensus_peers.cc:597] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 976359601ad74b149696dc7f555b550c. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:03.029893 1176 tablet_bootstrap.cc:492] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.030592 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_f99f927cdc404f66849fad37ee1f1445 [id=3637ceb2e2d8469ebc7c9f871c73c806]
I20260505 07:54:03.030915 1176 ts_tablet_manager.cc:1403] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.180s user 0.014s sys 0.010s
I20260505 07:54:03.033403 1172 tablet_bootstrap.cc:492] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.034075 1801 raft_consensus.cc:493] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.034260 1801 raft_consensus.cc:515] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.035069 1172 ts_tablet_manager.cc:1403] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.097s user 0.002s sys 0.005s
I20260505 07:54:03.035929 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4783dd0b2fbe40219852d850fc0d9a82" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.036279 921 raft_consensus.cc:2468] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:03.036785 991 leader_election.cc:304] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:03.037118 1798 raft_consensus.cc:2804] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.037352 1798 raft_consensus.cc:493] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.037576 1798 raft_consensus.cc:3060] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.037761 1801 leader_election.cc:290] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.038174 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397d33e908af4a0bb8dde26d273be01a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:03.039394 988 leader_election.cc:343] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.040243 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4783dd0b2fbe40219852d850fc0d9a82" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:03.041414 988 leader_election.cc:343] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.041620 1798 raft_consensus.cc:515] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.042673 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4783dd0b2fbe40219852d850fc0d9a82" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:03.042729 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4783dd0b2fbe40219852d850fc0d9a82" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.042896 921 raft_consensus.cc:3060] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.043210 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]
W20260505 07:54:03.043278 988 leader_election.cc:343] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.043951 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_043f3d0ae81c46a9a77b3264088fd1a9 [id=37b0844f588e413f99413fa009c1ef0c]
I20260505 07:54:03.044075 1176 raft_consensus.cc:359] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.044791 1172 raft_consensus.cc:359] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.045085 1176 raft_consensus.cc:385] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.045218 1798 leader_election.cc:290] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.045568 1176 raft_consensus.cc:740] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.047760 1172 raft_consensus.cc:385] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.046298 921 raft_consensus.cc:2468] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:03.045892 1176 consensus_queue.cc:260] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.048256 991 leader_election.cc:304] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.047851 1172 raft_consensus.cc:740] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.049446 1181 tablet_bootstrap.cc:492] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:03.049607 1181 ts_tablet_manager.cc:1403] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.274s user 0.096s sys 0.128s
I20260505 07:54:03.050384 1181 raft_consensus.cc:359] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.050585 1181 raft_consensus.cc:385] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.050675 1181 raft_consensus.cc:740] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.050940 1181 consensus_queue.cc:260] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.051261 1803 raft_consensus.cc:2804] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.051411 1803 raft_consensus.cc:697] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:03.051496 1181 ts_tablet_manager.cc:1434] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:03.051748 1803 consensus_queue.cc:237] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.053601 1176 ts_tablet_manager.cc:1434] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.022s user 0.001s sys 0.001s
I20260505 07:54:03.053890 1176 tablet_bootstrap.cc:492] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.054814 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_37add05ae60c4b40b655ffe723848dfd [id=3a9a2fa1381b40608a07038149eae25e]
I20260505 07:54:03.055311 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9 [id=3aa673bdfdb2432bb2174d4a520d7a9e]
I20260505 07:54:03.053035 1172 consensus_queue.cc:260] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.056185 1172 ts_tablet_manager.cc:1434] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260505 07:54:03.056982 1779 consensus_queue.cc:1048] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.057377 1779 raft_consensus.cc:493] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.057525 1181 tablet_bootstrap.cc:492] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:03.057531 1779 raft_consensus.cc:515] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.058108 1779 leader_election.cc:290] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.058658 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a06f77255544d7dabb71a4f70ada809" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.058933 1172 tablet_bootstrap.cc:492] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.060133 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_cc718440246d4ddfb49faf4602910188 [id=3ae3bc457dcb4db1b5f4360cb9e5c81c]
I20260505 07:54:03.060619 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]
I20260505 07:54:03.061774 1176 tablet_bootstrap.cc:654] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.063148 1172 tablet_bootstrap.cc:654] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.064319 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]
I20260505 07:54:03.065364 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a06f77255544d7dabb71a4f70ada809" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.066143 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
/tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:35401
--webserver_interface=127.0.105.60
--webserver_port=44877
--builtin_ntp_servers=127.0.105.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260505 07:54:03.068385 1052 raft_consensus.cc:2393] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:03.070019 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]
I20260505 07:54:03.074537 717 leader_election.cc:304] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.075176 1779 raft_consensus.cc:3060] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.077096 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_8c264d1fb97d491bb94ba720eef63e01 [id=4ca8dc50d71c42c6adbdedeb932d3040]
I20260505 07:54:03.077551 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_a86abb495238445381ac0202804b4ca0 [id=4d3b183ab74a4078beb7583c9e60e2c6]
I20260505 07:54:03.078259 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_063cc9f08ab54458b9e277ca65091b11 [id=4f094780dd764a01bb6ba8c7b08d6ef0]
I20260505 07:54:03.078454 1779 raft_consensus.cc:2749] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.079203 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]
I20260505 07:54:03.079799 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_6de81e84d3e44828a5b8026b551bdeb3 [id=509c7b9d25aa48af9dde3825cfe313d0]
I20260505 07:54:03.087829 1172 tablet_bootstrap.cc:492] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.088055 1172 ts_tablet_manager.cc:1403] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.002s
I20260505 07:54:03.089071 1172 raft_consensus.cc:359] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.089329 1172 raft_consensus.cc:385] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.089464 1172 raft_consensus.cc:740] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.089756 1172 consensus_queue.cc:260] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.090973 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]
I20260505 07:54:03.091742 1798 consensus_queue.cc:1048] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.091750 1512 consensus_queue.cc:1048] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.096107 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_f40f1996d8454947afd1ff280e2b5fc1 [id=541feb1163924c0ca3fb40a84ced3826]
I20260505 07:54:03.099119 1172 ts_tablet_manager.cc:1434] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260505 07:54:03.102895 1176 tablet_bootstrap.cc:492] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.103128 1176 ts_tablet_manager.cc:1403] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.049s user 0.003s sys 0.003s
I20260505 07:54:03.106658 1176 raft_consensus.cc:359] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.106964 1176 raft_consensus.cc:385] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.107116 1176 raft_consensus.cc:740] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.107406 1176 consensus_queue.cc:260] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.108137 1176 ts_tablet_manager.cc:1434] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:03.109459 1176 tablet_bootstrap.cc:492] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.110447 1649 raft_consensus.cc:493] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.113276 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_ceaf4d54d57542928716e3056eac2bde [id=54621fbfaf6a4b4a874207bd986b77be]
I20260505 07:54:03.113608 1172 tablet_bootstrap.cc:492] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.115376 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_89965306827247e99ff9f34d13e63c54 [id=567b03439f3a412aa6c1a4f4bac48f4b]
I20260505 07:54:03.119130 1172 tablet_bootstrap.cc:654] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.120445 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_9c422539fda7423c820e2450a3f7c8db [id=5993b31e5350440f9715f1e416b90eb2]
I20260505 07:54:03.120635 1649 raft_consensus.cc:515] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:03.119269 1317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.60:35401, user_credentials={real_user=slave}} blocked reactor thread for 51785.6us
I20260505 07:54:03.122710 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_201f8409684e49be8bc5cd5249606250 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1]
I20260505 07:54:03.123356 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]
I20260505 07:54:03.124895 1176 tablet_bootstrap.cc:654] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.127171 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_5f0d98b4110b4596a6de3749d10302c9 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]
I20260505 07:54:03.127863 1649 leader_election.cc:290] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:03.128935 718 leader_election.cc:343] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 1ef4f516e64a44fd8cd33674fb808067
I20260505 07:54:03.130232 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_1e6b1ec4673c45e6b7b6c2696722d84b [id=5d0d6b47936a454c9a93cc8af031807b]
I20260505 07:54:03.130779 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_de416cf2bc944e20bb8b3d84e91585c5 [id=5dd5f713f7a24f85a694f2af5171dba3]
I20260505 07:54:03.131512 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_0410f2bbd0504738ba4aa1f7412e3405 [id=617f784f561c4c9c93be1bdd500fc4d9]
W20260505 07:54:03.132117 717 consensus_peers.cc:597] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a79578757824bfc9df129e3475e7af3. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:03.132452 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]
I20260505 07:54:03.134069 1649 raft_consensus.cc:493] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.134158 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_4ab0ac01b8bf49ef96683dca41dfadd4 [id=6cb8bc8043474bad9f4278b7eef7c71a]
I20260505 07:54:03.134268 1649 raft_consensus.cc:515] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.134586 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]
I20260505 07:54:03.134963 1649 leader_election.cc:290] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.135010 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_ed41728f809d4b0eb56670c1410d14ac [id=6e6683c2d09e4532aa3a56246c56bfd4]
I20260505 07:54:03.135679 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840eebf9e7404bf7807b7e51996957e5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.135895 921 raft_consensus.cc:2468] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.136471 718 leader_election.cc:304] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
W20260505 07:54:03.137176 718 consensus_peers.cc:597] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e707143a4e64725aa66db05b24999a6. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:03.137413 1649 raft_consensus.cc:2804] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.137511 1649 raft_consensus.cc:493] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.137615 1649 raft_consensus.cc:3060] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.139945 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ef4f516e64a44fd8cd33674fb808067" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.140146 1053 raft_consensus.cc:2468] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.127609 1804 consensus_queue.cc:1048] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260505 07:54:03.140376 1649 raft_consensus.cc:515] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.141868 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840eebf9e7404bf7807b7e51996957e5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.142040 921 raft_consensus.cc:3060] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.144397 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840eebf9e7404bf7807b7e51996957e5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:03.144721 717 leader_election.cc:304] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:03.144972 1512 raft_consensus.cc:2804] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.145071 1512 raft_consensus.cc:493] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.145146 1512 raft_consensus.cc:3060] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.146411 1054 raft_consensus.cc:3060] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.147176 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840eebf9e7404bf7807b7e51996957e5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.147801 1512 raft_consensus.cc:515] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.148214 1181 tablet_bootstrap.cc:654] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.148344 1176 tablet_bootstrap.cc:492] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.148392 1512 leader_election.cc:290] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.148492 1176 ts_tablet_manager.cc:1403] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.039s user 0.005s sys 0.001s
I20260505 07:54:03.149269 1176 raft_consensus.cc:359] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.149421 1176 raft_consensus.cc:385] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.149511 1176 raft_consensus.cc:740] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
W20260505 07:54:03.149595 718 leader_election.cc:343] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 1ef4f516e64a44fd8cd33674fb808067
I20260505 07:54:03.144822 921 raft_consensus.cc:2468] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:03.150154 718 leader_election.cc:304] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.150455 1512 raft_consensus.cc:2804] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.151034 1512 raft_consensus.cc:697] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:03.151299 1054 raft_consensus.cc:2468] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:03.151372 1512 consensus_queue.cc:237] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.151536 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ef4f516e64a44fd8cd33674fb808067" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:03.151800 1053 raft_consensus.cc:3060] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.152225 1649 leader_election.cc:290] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.154372 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_394242d5ee184ae2b22cc355a2160fb9 [id=6e71d3f928fc45c191f64a945980a107]
I20260505 07:54:03.155269 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_6b9d99bdf8624bba90ba61af22f36dc2 [id=6fa686898f454d2aa691814ad1284d0f]
I20260505 07:54:03.155759 1053 raft_consensus.cc:2468] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:03.156263 717 leader_election.cc:304] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:03.156656 1649 raft_consensus.cc:2804] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.157058 1649 raft_consensus.cc:697] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:03.157359 1649 consensus_queue.cc:237] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.158433 1172 tablet_bootstrap.cc:492] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.158644 1172 ts_tablet_manager.cc:1403] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.045s user 0.005s sys 0.000s
I20260505 07:54:03.149709 1176 consensus_queue.cc:260] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.160765 1176 ts_tablet_manager.cc:1434] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260505 07:54:03.161775 1176 tablet_bootstrap.cc:492] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.163702 1176 tablet_bootstrap.cc:654] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.164846 1172 raft_consensus.cc:359] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.165175 1172 raft_consensus.cc:385] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.165313 1172 raft_consensus.cc:740] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.165696 1172 consensus_queue.cc:260] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.165795 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_d4c7087457154e89b20abe713355e422 [id=7099b9d0095f4bf880cff569eeeb5d32]
I20260505 07:54:03.166241 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_ab5be55234c742aea9421c70e2dad3bb [id=70c9d1bf26e34dd69abefcef94c785c5]
I20260505 07:54:03.167264 1172 ts_tablet_manager.cc:1434] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.008s user 0.002s sys 0.001s
I20260505 07:54:03.168056 1779 raft_consensus.cc:493] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.168241 1779 raft_consensus.cc:515] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.168936 1779 leader_election.cc:290] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.169608 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78a6470720984761a8606abeafbef992" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.169587 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78a6470720984761a8606abeafbef992" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.169860 1053 raft_consensus.cc:2393] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:03.170109 1172 tablet_bootstrap.cc:492] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.170189 1804 raft_consensus.cc:493] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.170327 1804 raft_consensus.cc:515] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.170533 717 leader_election.cc:304] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.170948 1779 raft_consensus.cc:3060] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.170965 1804 leader_election.cc:290] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.171690 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17327cee69a24e0cb493ff55f948bf6c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.171864 920 raft_consensus.cc:2468] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:03.172026 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17327cee69a24e0cb493ff55f948bf6c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.172581 991 leader_election.cc:304] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
W20260505 07:54:03.173010 988 leader_election.cc:343] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.173312 1804 raft_consensus.cc:2804] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.173385 1804 raft_consensus.cc:493] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.173473 1804 raft_consensus.cc:3060] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.173600 1779 raft_consensus.cc:2749] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.175328 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_02afe59354994bffb24b190e9a7abf85 [id=7579207045ec4220b966af9016a8f050]
I20260505 07:54:03.177354 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]
I20260505 07:54:03.177974 1804 raft_consensus.cc:515] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.178992 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17327cee69a24e0cb493ff55f948bf6c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:03.179457 988 leader_election.cc:343] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.180305 1804 leader_election.cc:290] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.181424 1181 tablet_bootstrap.cc:492] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:03.181597 1181 ts_tablet_manager.cc:1403] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.126s user 0.007s sys 0.001s
I20260505 07:54:03.182438 1181 raft_consensus.cc:359] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.182614 1181 raft_consensus.cc:385] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.182708 1181 raft_consensus.cc:740] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.182958 1181 consensus_queue.cc:260] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.183696 1181 ts_tablet_manager.cc:1434] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:03.183979 1181 tablet_bootstrap.cc:492] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:03.185580 1181 tablet_bootstrap.cc:654] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.187345 1176 tablet_bootstrap.cc:492] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.187492 1176 ts_tablet_manager.cc:1403] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.003s
I20260505 07:54:03.188310 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17327cee69a24e0cb493ff55f948bf6c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.188517 921 raft_consensus.cc:3060] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.188576 1172 tablet_bootstrap.cc:654] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.190318 1176 raft_consensus.cc:359] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.191500 921 raft_consensus.cc:2468] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:03.192867 991 leader_election.cc:304] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.193003 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_21673305a26a4342a0be11d98e875447 [id=76c4943b63be49fdaf6ee51abe09b9b3]
I20260505 07:54:03.193272 1804 raft_consensus.cc:2804] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.193420 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95]
I20260505 07:54:03.193665 1804 raft_consensus.cc:697] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:03.193951 1804 consensus_queue.cc:237] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.194546 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]
I20260505 07:54:03.195001 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_55fc26564d414d72992d4651acdbc6bb [id=77bd7e34452244009c813333c4c32ceb]
I20260505 07:54:03.190569 1176 raft_consensus.cc:385] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.195678 1176 raft_consensus.cc:740] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.195999 1176 consensus_queue.cc:260] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.196573 1176 ts_tablet_manager.cc:1434] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20260505 07:54:03.198734 1176 tablet_bootstrap.cc:492] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.199338 1580 raft_consensus.cc:493] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.199492 1580 raft_consensus.cc:515] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.200111 1580 leader_election.cc:290] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.200534 1176 tablet_bootstrap.cc:654] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.200950 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4662d4a3d88d4c2883a35048d6e11eb9" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.201210 782 raft_consensus.cc:2393] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:03.203087 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]
I20260505 07:54:03.204502 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_79ed240074b343fabbdd206872328277 [id=7cbbeae00a3d4f85bcaef70b7aaa69de]
I20260505 07:54:03.205694 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_d2caf74ce2d94a2ea94685efc828218b [id=7ce7150d6a004dcb94857240940133de]
I20260505 07:54:03.207677 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4662d4a3d88d4c2883a35048d6e11eb9" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.208163 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_948a14e895244fb893e72073a38d1e14 [id=85d8a9438bef4555b0ada2eceffefc0d]
I20260505 07:54:03.208277 991 leader_election.cc:304] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.208740 1580 raft_consensus.cc:3060] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.209475 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f]
I20260505 07:54:03.210078 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_fa6246cc3ccd4d7082becde923f88828 [id=860ab29ce1a44ad1b2480dbdb2b40000]
I20260505 07:54:03.211839 1580 raft_consensus.cc:2749] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.212241 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c [id=8a910f6a553a47ea87ec170e376982d1]
W20260505 07:54:03.214962 718 consensus_peers.cc:597] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 872362aecf6a44b59e586b1fa7637c8b. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:03.218500 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_ec95dd1d091844c3adb51935b2e48795 [id=8b6dda11eb884c7da54b226291f403c5]
I20260505 07:54:03.219219 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_d17cd5686eb94c3b8993d4310f0d31f7 [id=8c0e89be595c4eb2b4d4103121cdccf0]
W20260505 07:54:03.220942 718 consensus_peers.cc:597] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d6494590052459485059f5dce46f34f. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:03.221786 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_f3a7ead203124b49a437c53941ec4d58 [id=8ebfbe0aa09a43bdbb4b8f323658b3b1]
I20260505 07:54:03.222733 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]
I20260505 07:54:03.223654 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_97ee3338c1344152a0d674d42c76d5be [id=908887d51e8949ac81cebbcb0c6ddb6d]
I20260505 07:54:03.224747 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]
I20260505 07:54:03.225901 1176 tablet_bootstrap.cc:492] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.226065 1176 ts_tablet_manager.cc:1403] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.005s
I20260505 07:54:03.226810 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_7001b868854949a5b9bee0ba45c2f522 [id=978b4030e04a4e32938aa646b1e85e6c]
I20260505 07:54:03.226914 1176 raft_consensus.cc:359] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.227094 1176 raft_consensus.cc:385] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.227195 1176 raft_consensus.cc:740] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.228041 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_359f92a4ff554e3e80468261b0e86966 [id=9930805fdf364f25aea11f859b50bc71]
I20260505 07:54:03.230204 1569 raft_consensus.cc:493] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.230355 1569 raft_consensus.cc:515] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.230979 1569 leader_election.cc:290] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:03.231276 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]
I20260505 07:54:03.232203 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2494780bdd4b6dae6fbdf71125da88" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:03.232837 854 leader_election.cc:343] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.234277 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_1f4b1af520d64a108ff4289226a1561d [id=a238daba389c412ea67318d99cfb5559]
I20260505 07:54:03.234686 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_9cfdbaa4e1834e3fad8d09804f626d78 [id=a3258edb643a47ecb936efc0fc489743]
I20260505 07:54:03.237694 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_876e4bf0b9054b8e836299103ac03ef7 [id=a3b569958e05423588c9949eb7fe34a6]
I20260505 07:54:03.239064 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]
I20260505 07:54:03.239784 1176 consensus_queue.cc:260] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.241156 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_06ea1b3a616b4c5aa573a1c654538879 [id=a648fb7ba63246c38e2a9f120d6d0396]
I20260505 07:54:03.242151 1172 tablet_bootstrap.cc:492] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.242110 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2494780bdd4b6dae6fbdf71125da88" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.242296 1172 ts_tablet_manager.cc:1403] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.072s user 0.010s sys 0.001s
I20260505 07:54:03.242595 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_f93a81c5dfe740819f0f6ea5bf7f1067 [id=a8a6d42a6b6748d590ef5949a9d1e650]
W20260505 07:54:03.242729 856 leader_election.cc:343] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.242884 856 leader_election.cc:304] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.243121 1172 raft_consensus.cc:359] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.243239 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]
I20260505 07:54:03.243566 1176 ts_tablet_manager.cc:1434] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.017s user 0.000s sys 0.002s
I20260505 07:54:03.244588 1176 tablet_bootstrap.cc:492] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.245415 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_0393e87d0a314ee898b11935e2f6666d [id=ab93bfbabbad4fad893f7a6dd595540f]
I20260505 07:54:03.243527 1569 raft_consensus.cc:2749] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.245801 1172 raft_consensus.cc:385] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.246783 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_54f371d41b5e4a54b08bb82a834ac0af [id=abda2c3e5e9d4c64b0d08510cb3d8aed]
I20260505 07:54:03.246937 1668 raft_consensus.cc:493] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 23c57a2c927c4c54bccecfeb3a775311)
I20260505 07:54:03.247114 1668 raft_consensus.cc:515] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:54:03.247995 1668 leader_election.cc:290] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565), 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401)
I20260505 07:54:03.248417 1172 raft_consensus.cc:740] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.248672 1172 consensus_queue.cc:260] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.249756 1176 tablet_bootstrap.cc:654] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.251322 1172 ts_tablet_manager.cc:1434] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260505 07:54:03.252308 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_95fae21889c84b61ad67610d209031b2 [id=b131fb4265684cd2b6506a7abeb29871]
I20260505 07:54:03.253849 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]
I20260505 07:54:03.256122 1172 tablet_bootstrap.cc:492] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.258030 1172 tablet_bootstrap.cc:654] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.259119 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]
I20260505 07:54:03.260115 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_6ddd9e0382d845d1b89edce006361e1a [id=ba1b026067294e0cae324fb994815400]
I20260505 07:54:03.261044 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_881e42cce93c46cb8c6b2ff28d713b85 [id=bc1adc573d4c43afa8c5bdd900e53d65]
I20260505 07:54:03.262095 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_7e65acc795454f48ad6b8d0dc5dc163d [id=bccdbf7ecfdd40c59dc52c4562f88728]
I20260505 07:54:03.266350 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_59819a75045240de818a627f6036f8e3 [id=c019a4cae5d0458b85da8106e6819924]
I20260505 07:54:03.268031 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_6cb216f92da74a78acd72a6132b94c1a [id=c0306ceddf024a1589f83f390b38c602]
I20260505 07:54:03.268811 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_f3fff69ccbc54a58bdad7a8b5489d525 [id=c0d6550f259540e2943e2378fc4bfde5]
I20260505 07:54:03.269798 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5 [id=c10b484439c443368141ceb4cf43365f]
I20260505 07:54:03.270757 1172 tablet_bootstrap.cc:492] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.271026 1172 ts_tablet_manager.cc:1403] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.005s
I20260505 07:54:03.271435 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]
I20260505 07:54:03.274861 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]
I20260505 07:54:03.276048 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_40605e9a72634999bc80f464976c5dac [id=c1d7934526274ef09104dbff5eaacb1e]
W20260505 07:54:03.276295 444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111) [suppressed 7 similar messages]
I20260505 07:54:03.277323 1172 raft_consensus.cc:359] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.278702 1172 raft_consensus.cc:385] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.279343 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_dedf48e6bbc941f6b329f367499612f2 [id=c4d8c3d8aa554154a7015e6ff3d7b38c]
W20260505 07:54:03.279639 444 leader_election.cc:336] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401): Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:54:03.278929 1172 raft_consensus.cc:740] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.280362 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_ad8e6e942c73439fb64536b3ab1edc9c [id=c7cad0401e40485e9ed03160b9bed704]
I20260505 07:54:03.281224 1172 consensus_queue.cc:260] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.283347 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]
I20260505 07:54:03.283330 1172 ts_tablet_manager.cc:1434] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260505 07:54:03.284287 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_f8b3c6c6274f468fb9f56dad70e9043c [id=c91f3cdfd7534217ba6f4c985afd18e4]
I20260505 07:54:03.284839 1172 tablet_bootstrap.cc:492] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.286343 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_a31277b1c1ca4863976cb9726e83f13a [id=d4aea2d5304d4113ab26a474454b05de]
I20260505 07:54:03.289240 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_e7ad0f00a4fc456e944f70e21d505fd5 [id=d5242964630749eeb15c3c7c9b0f3686]
I20260505 07:54:03.289877 1569 raft_consensus.cc:493] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.290076 1569 raft_consensus.cc:515] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.290376 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_633da5b39da34f5c88163c9936fd0212 [id=d53c653124364ea5acc357977c9db864]
I20260505 07:54:03.290673 1569 leader_election.cc:290] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:03.290987 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_8f7c0f8991b1415abce7b31407477b52 [id=d6192f5cec174951bf17af2e65d01f79]
I20260505 07:54:03.291179 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0459f2d45a2d44f1a73e0a0fff53866c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.291420 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0459f2d45a2d44f1a73e0a0fff53866c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:03.291806 854 leader_election.cc:343] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:03.291965 856 leader_election.cc:343] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.292069 856 leader_election.cc:304] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.292536 1569 raft_consensus.cc:2749] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.293210 1172 tablet_bootstrap.cc:654] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.293308 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_a8605cf7bc3b44f39a895bfb7ad9934b [id=daf1e66deafd417787948bb18d615391]
I20260505 07:54:03.295178 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_d01b41eea75c46339079b05376f7d496 [id=dcf4d78fda334683bf51e07167c9999a]
I20260505 07:54:03.297895 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_8564f8e67e6f4995b312673dd0d63f52 [id=de1506e299994d1fbe31033d301d4533]
I20260505 07:54:03.299335 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_7ed6d926da574429be8db03bde4b4221 [id=e52c3ee18ce644d2b988c2e70c6307d7]
I20260505 07:54:03.302302 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_5e1ab4e85d4b46d8a5d98434da77c9a8 [id=e86701e2ef8c4075877c58927318622f]
I20260505 07:54:03.302418 1172 tablet_bootstrap.cc:492] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.302695 1172 ts_tablet_manager.cc:1403] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.001s
W20260505 07:54:03.304237 991 consensus_peers.cc:597] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 202a237ca580482c98803683a77d96d2. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:03.304471 1172 raft_consensus.cc:359] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.304821 1172 raft_consensus.cc:385] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.305032 1172 raft_consensus.cc:740] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.305234 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_eb1e569ff0a74997b956acbad82f6c49 [id=e996e55aed7c4c08bb2fd8a6fc358155]
I20260505 07:54:03.305318 1181 tablet_bootstrap.cc:492] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:03.305487 1181 ts_tablet_manager.cc:1403] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.122s user 0.004s sys 0.005s
I20260505 07:54:03.305996 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_abc41dd87e7a45c1b52bf036a9dfab27 [id=eb2e83fd347c4d2583a6fdf9bac20d53]
I20260505 07:54:03.305661 1172 consensus_queue.cc:260] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.306257 1181 raft_consensus.cc:359] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.306492 1181 raft_consensus.cc:385] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.306582 1181 raft_consensus.cc:740] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.306826 1181 consensus_queue.cc:260] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.307310 1181 ts_tablet_manager.cc:1434] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:03.307358 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]
I20260505 07:54:03.308187 1181 tablet_bootstrap.cc:492] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:03.309459 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]
I20260505 07:54:03.309983 1181 tablet_bootstrap.cc:654] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.309957 1172 ts_tablet_manager.cc:1434] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260505 07:54:03.310786 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]
I20260505 07:54:03.311746 1172 tablet_bootstrap.cc:492] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.312635 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_657cc0ba6f494479bd433c8c0279c734 [id=f191fbb56e234f10a9f94b2b998781bd]
I20260505 07:54:03.313828 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_f75ef4b5e6ae433f8c6bee72fc30eaf2 [id=f35c58e2d1f54d3ba63a303a0648cebf]
I20260505 07:54:03.314998 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_fcec571b3c7140669fe8d8c983cd616f [id=f47edf0bb7a14132a37d1f820082f0b9]
I20260505 07:54:03.315357 1512 raft_consensus.cc:493] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.315605 1512 raft_consensus.cc:515] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.316314 1512 leader_election.cc:290] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.317459 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323269e708964848a94c64e6a781a860" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:03.318471 717 leader_election.cc:343] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.318814 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323269e708964848a94c64e6a781a860" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.319096 919 raft_consensus.cc:2468] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.319546 718 leader_election.cc:304] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.320014 1645 raft_consensus.cc:2804] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.320106 1645 raft_consensus.cc:493] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.320215 1645 raft_consensus.cc:3060] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.320353 1172 tablet_bootstrap.cc:654] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.321521 1181 tablet_bootstrap.cc:492] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:03.321646 1181 ts_tablet_manager.cc:1403] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.001s
I20260505 07:54:03.322364 1181 raft_consensus.cc:359] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.322520 1181 raft_consensus.cc:385] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.322608 1181 raft_consensus.cc:740] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.322921 1181 consensus_queue.cc:260] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.323011 1645 raft_consensus.cc:515] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.323525 1181 ts_tablet_manager.cc:1434] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:03.325160 1645 leader_election.cc:290] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.325882 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323269e708964848a94c64e6a781a860" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.326074 921 raft_consensus.cc:3060] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.326067 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "323269e708964848a94c64e6a781a860" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:03.326231 1055 raft_consensus.cc:3060] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.329090 921 raft_consensus.cc:2468] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:03.329258 1055 raft_consensus.cc:2468] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:03.329883 1181 tablet_bootstrap.cc:492] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:03.331676 1181 tablet_bootstrap.cc:654] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.329689 718 leader_election.cc:304] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.330427 1512 consensus_queue.cc:1048] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.333053 1512 raft_consensus.cc:2804] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.333220 1512 raft_consensus.cc:697] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:03.333477 1512 consensus_queue.cc:237] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.341116 1181 tablet_bootstrap.cc:492] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:03.341256 1181 ts_tablet_manager.cc:1403] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260505 07:54:03.341910 1181 raft_consensus.cc:359] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.342082 1181 raft_consensus.cc:385] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.342149 1181 raft_consensus.cc:740] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.348640 1181 consensus_queue.cc:260] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.349773 1181 ts_tablet_manager.cc:1434] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260505 07:54:03.350066 1181 tablet_bootstrap.cc:492] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:03.352010 1176 tablet_bootstrap.cc:492] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.352187 1176 ts_tablet_manager.cc:1403] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.108s user 0.005s sys 0.021s
I20260505 07:54:03.352226 1181 tablet_bootstrap.cc:654] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.353119 1176 raft_consensus.cc:359] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.353317 1176 raft_consensus.cc:385] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.353468 1176 raft_consensus.cc:740] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.353698 1176 consensus_queue.cc:260] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.354157 1176 ts_tablet_manager.cc:1434] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:03.354848 1176 tablet_bootstrap.cc:492] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.356593 1176 tablet_bootstrap.cc:654] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.368916 1181 tablet_bootstrap.cc:492] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:03.369180 1181 ts_tablet_manager.cc:1403] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260505 07:54:03.370467 1181 raft_consensus.cc:359] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.370741 1181 raft_consensus.cc:385] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.370947 1181 raft_consensus.cc:740] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.371379 1181 consensus_queue.cc:260] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.372840 1176 tablet_bootstrap.cc:492] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.372978 1176 ts_tablet_manager.cc:1403] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.003s
I20260505 07:54:03.373549 1181 ts_tablet_manager.cc:1434] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:03.373790 1176 raft_consensus.cc:359] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.373965 1176 raft_consensus.cc:385] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.374047 1176 raft_consensus.cc:740] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.374274 1176 consensus_queue.cc:260] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.374749 1176 ts_tablet_manager.cc:1434] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:03.375072 1176 tablet_bootstrap.cc:492] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.376298 1181 tablet_bootstrap.cc:492] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:03.376816 1176 tablet_bootstrap.cc:654] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:03.380175 854 consensus_peers.cc:597] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:03.381793 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_659ddda68e194bb7ae10409bbef074c6 [id=f6e2d9ac0e374944988e8ef7f88fa4be]
I20260505 07:54:03.384598 1181 tablet_bootstrap.cc:654] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.388324 1569 consensus_queue.cc:1048] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.392133 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_94e6f1ee4ab947d8bc9864ab0c8bd524 [id=f7b2051b32e240c1856be3f0802b2f3e]
I20260505 07:54:03.394244 1176 tablet_bootstrap.cc:492] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.394423 1176 ts_tablet_manager.cc:1403] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260505 07:54:03.395269 1176 raft_consensus.cc:359] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.395478 1176 raft_consensus.cc:385] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.395568 1176 raft_consensus.cc:740] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.395947 1176 consensus_queue.cc:260] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.396452 1176 ts_tablet_manager.cc:1434] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:03.396780 1176 tablet_bootstrap.cc:492] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.398479 1176 tablet_bootstrap.cc:654] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.399309 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_55f3a1ca8c53455993b486b0798f0425 [id=fb5d62bb12c5499e8953cb3b431867a9]
I20260505 07:54:03.401041 1804 raft_consensus.cc:493] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.401202 1804 raft_consensus.cc:515] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.401786 1804 leader_election.cc:290] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.402770 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.402784 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.403000 919 raft_consensus.cc:2468] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:03.403380 991 leader_election.cc:304] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
W20260505 07:54:03.403806 988 leader_election.cc:343] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.404136 1804 raft_consensus.cc:2804] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.404245 1804 raft_consensus.cc:493] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.404331 1804 raft_consensus.cc:3060] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.408654 1176 tablet_bootstrap.cc:492] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.408814 1176 ts_tablet_manager.cc:1403] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.002s
I20260505 07:54:03.409546 1176 raft_consensus.cc:359] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.409591 1580 consensus_queue.cc:1048] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.409706 1176 raft_consensus.cc:385] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.409786 1176 raft_consensus.cc:740] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.410020 1176 consensus_queue.cc:260] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.410566 1176 ts_tablet_manager.cc:1434] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:03.410851 1176 tablet_bootstrap.cc:492] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.412384 1176 tablet_bootstrap.cc:654] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.413820 1804 raft_consensus.cc:515] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.414330 1804 leader_election.cc:290] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.414705 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:03.415637 988 leader_election.cc:343] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.415978 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.416213 919 raft_consensus.cc:3060] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.419420 919 raft_consensus.cc:2468] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:03.419637 1804 raft_consensus.cc:493] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.419902 1804 raft_consensus.cc:515] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.420893 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70404184502a4394a737518220f2124f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.421098 779 raft_consensus.cc:2468] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:03.421550 988 leader_election.cc:304] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.421909 1580 raft_consensus.cc:2804] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.422030 1580 raft_consensus.cc:493] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.422114 1580 raft_consensus.cc:3060] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.425909 1596 consensus_queue.cc:1048] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.428105 1176 tablet_bootstrap.cc:492] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.428254 1176 ts_tablet_manager.cc:1403] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20260505 07:54:03.428979 1176 raft_consensus.cc:359] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.429158 1176 raft_consensus.cc:385] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.429229 1176 raft_consensus.cc:740] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.429450 1176 consensus_queue.cc:260] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.429934 1176 ts_tablet_manager.cc:1434] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:03.430234 1176 tablet_bootstrap.cc:492] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.431979 1176 tablet_bootstrap.cc:654] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.434140 1580 raft_consensus.cc:515] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.434864 1580 leader_election.cc:290] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.435887 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70404184502a4394a737518220f2124f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:03.436093 780 raft_consensus.cc:3060] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.440866 780 raft_consensus.cc:2468] T 70404184502a4394a737518220f2124f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:03.441185 991 consensus_peers.cc:597] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1838f89dc14a47f99f95f4cc5fdefd70. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:03.441344 988 leader_election.cc:304] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.441813 1580 raft_consensus.cc:2804] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.441999 1580 raft_consensus.cc:697] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:03.446326 1176 tablet_bootstrap.cc:492] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.446584 1176 ts_tablet_manager.cc:1403] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260505 07:54:03.425424 1569 consensus_queue.cc:1048] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.448446 1176 raft_consensus.cc:359] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.434401 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70404184502a4394a737518220f2124f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.448742 1176 raft_consensus.cc:385] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.448838 921 raft_consensus.cc:2468] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:03.435168 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70404184502a4394a737518220f2124f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.448978 1176 raft_consensus.cc:740] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.449143 920 raft_consensus.cc:3060] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.449412 1176 consensus_queue.cc:260] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.450618 1176 ts_tablet_manager.cc:1434] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260505 07:54:03.451327 1176 tablet_bootstrap.cc:492] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.453490 920 raft_consensus.cc:2468] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:03.456857 1569 raft_consensus.cc:493] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.457053 1569 raft_consensus.cc:515] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.457778 1569 leader_election.cc:290] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:03.457917 1176 tablet_bootstrap.cc:654] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.458575 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af9def97a454f3f9b63d637db42c110" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:03.459177 854 leader_election.cc:343] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.463706 1580 consensus_queue.cc:237] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.465044 991 leader_election.cc:304] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.465946 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af9def97a454f3f9b63d637db42c110" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.466383 1798 raft_consensus.cc:2804] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.466562 1798 raft_consensus.cc:697] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:03.466902 1798 consensus_queue.cc:237] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:03.469681 856 leader_election.cc:343] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.469883 856 leader_election.cc:304] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.470367 1569 raft_consensus.cc:2749] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.471004 1804 leader_election.cc:290] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.478590 1176 tablet_bootstrap.cc:492] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.479086 1176 ts_tablet_manager.cc:1403] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.015s
I20260505 07:54:03.480839 1176 raft_consensus.cc:359] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.481187 1176 raft_consensus.cc:385] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.481315 1176 raft_consensus.cc:740] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.481727 1176 consensus_queue.cc:260] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.482825 1569 consensus_queue.cc:1048] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.482897 1176 ts_tablet_manager.cc:1434] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260505 07:54:03.483479 1176 tablet_bootstrap.cc:492] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.484807 1569 raft_consensus.cc:493] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.484982 1569 raft_consensus.cc:515] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.485821 1569 leader_election.cc:290] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:03.486545 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0baf83b155494fddb3c25144608706f7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.487272 1176 tablet_bootstrap.cc:654] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:03.487339 854 leader_election.cc:343] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.488462 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0baf83b155494fddb3c25144608706f7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:03.489380 856 leader_election.cc:343] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.489528 856 leader_election.cc:304] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.490080 1569 raft_consensus.cc:2749] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.491843 1645 consensus_queue.cc:1048] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260505 07:54:03.509477 1645 raft_consensus.cc:493] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.509599 1176 tablet_bootstrap.cc:492] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.509877 1176 ts_tablet_manager.cc:1403] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.027s user 0.007s sys 0.012s
I20260505 07:54:03.509835 1645 raft_consensus.cc:515] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.511386 1176 raft_consensus.cc:359] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.511726 1176 raft_consensus.cc:385] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.511910 1176 raft_consensus.cc:740] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.512362 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.512280 1176 consensus_queue.cc:260] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.514236 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.514462 1176 ts_tablet_manager.cc:1434] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260505 07:54:03.514578 921 raft_consensus.cc:2393] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:03.515005 1176 tablet_bootstrap.cc:492] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.515516 718 leader_election.cc:304] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.516151 1512 raft_consensus.cc:3060] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.518970 1176 tablet_bootstrap.cc:654] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.519186 1569 raft_consensus.cc:493] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.519369 1569 raft_consensus.cc:515] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.520205 1569 leader_election.cc:290] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:03.520581 1649 raft_consensus.cc:493] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.520762 1649 raft_consensus.cc:515] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.521420 1649 leader_election.cc:290] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.522286 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff7ed926a3ff4e7993864de734ca3c50" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.522464 781 raft_consensus.cc:2468] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:03.523030 854 leader_election.cc:304] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.524688 1512 raft_consensus.cc:2749] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.525418 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff7ed926a3ff4e7993864de734ca3c50" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.526386 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5683d7366de84653ace56f0edcd3818c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.526636 921 raft_consensus.cc:2468] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.527526 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5683d7366de84653ace56f0edcd3818c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:03.528530 856 leader_election.cc:343] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.529014 1569 raft_consensus.cc:2804] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.529140 1569 raft_consensus.cc:493] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.529253 1569 raft_consensus.cc:3060] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.534204 1569 raft_consensus.cc:515] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.535527 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff7ed926a3ff4e7993864de734ca3c50" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:03.536201 856 leader_election.cc:343] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.537546 1569 leader_election.cc:290] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:03.528108 718 leader_election.cc:304] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.530292 1176 tablet_bootstrap.cc:492] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.537352 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff7ed926a3ff4e7993864de734ca3c50" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:03.538375 1176 ts_tablet_manager.cc:1403] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.001s
I20260505 07:54:03.538584 1779 raft_consensus.cc:2804] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.538695 1779 raft_consensus.cc:493] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.538925 1779 raft_consensus.cc:3060] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.540928 1176 raft_consensus.cc:359] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.538431 781 raft_consensus.cc:3060] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.541285 1176 raft_consensus.cc:385] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.541484 1176 raft_consensus.cc:740] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
W20260505 07:54:03.541744 717 leader_election.cc:343] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.541980 1176 consensus_queue.cc:260] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.542522 1649 consensus_queue.cc:1048] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260505 07:54:03.549263 1779 raft_consensus.cc:515] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.550383 1779 leader_election.cc:290] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.551397 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5683d7366de84653ace56f0edcd3818c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:03.552842 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5683d7366de84653ace56f0edcd3818c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.553078 921 raft_consensus.cc:3060] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.553304 1176 ts_tablet_manager.cc:1434] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.015s user 0.004s sys 0.005s
I20260505 07:54:03.553938 1176 tablet_bootstrap.cc:492] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:54:03.557381 717 leader_election.cc:343] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.558332 1176 tablet_bootstrap.cc:654] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.558513 781 raft_consensus.cc:2468] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:03.559237 854 leader_election.cc:304] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
W20260505 07:54:03.559998 718 consensus_peers.cc:597] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bd581a6799a14a8dafb80d411e8936f3. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:03.560895 1848 raft_consensus.cc:2804] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.562189 921 raft_consensus.cc:2468] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:03.563014 718 leader_election.cc:304] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.563503 1864 raft_consensus.cc:2804] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.563881 1804 raft_consensus.cc:493] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.564102 1804 raft_consensus.cc:515] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.565457 1804 leader_election.cc:290] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.565333 1864 raft_consensus.cc:697] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:03.565505 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accb2dedc30e494ebd22c33594cee661" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.565711 918 raft_consensus.cc:2468] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:03.566094 991 leader_election.cc:304] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:03.566442 1804 raft_consensus.cc:2804] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.566584 1804 raft_consensus.cc:493] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.566767 1804 raft_consensus.cc:3060] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.566241 1864 consensus_queue.cc:237] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.567267 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accb2dedc30e494ebd22c33594cee661" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:03.567863 988 leader_election.cc:343] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.569792 1804 raft_consensus.cc:515] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.570736 1804 leader_election.cc:290] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.571201 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accb2dedc30e494ebd22c33594cee661" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.571761 1779 consensus_queue.cc:1048] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.572192 920 raft_consensus.cc:3060] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.573076 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accb2dedc30e494ebd22c33594cee661" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:03.579319 988 leader_election.cc:343] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.579738 1848 raft_consensus.cc:697] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:03.580212 1848 consensus_queue.cc:237] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.537856 1645 leader_election.cc:290] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.587265 1176 tablet_bootstrap.cc:492] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.587903 1176 ts_tablet_manager.cc:1403] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.034s user 0.010s sys 0.002s
I20260505 07:54:03.589862 1176 raft_consensus.cc:359] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.590330 1176 raft_consensus.cc:385] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.590517 1176 raft_consensus.cc:740] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.590893 1176 consensus_queue.cc:260] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.592917 1176 ts_tablet_manager.cc:1434] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260505 07:54:03.593557 1596 raft_consensus.cc:493] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.593731 1596 raft_consensus.cc:515] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.593945 1866 consensus_queue.cc:1048] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.594301 1176 tablet_bootstrap.cc:492] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.594506 1596 leader_election.cc:290] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.595209 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d1ce00e487d4702a6affcad57a7fbe1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.595523 917 raft_consensus.cc:2468] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:03.596024 991 leader_election.cc:304] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:03.597496 1596 raft_consensus.cc:2804] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.597641 1596 raft_consensus.cc:493] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.597738 1596 raft_consensus.cc:3060] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.598289 1176 tablet_bootstrap.cc:654] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.602345 1596 raft_consensus.cc:515] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.603854 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d1ce00e487d4702a6affcad57a7fbe1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.604048 917 raft_consensus.cc:3060] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.607482 917 raft_consensus.cc:2468] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:03.608148 991 leader_election.cc:304] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:03.608472 1803 raft_consensus.cc:2804] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.608616 1803 raft_consensus.cc:697] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:03.608925 1803 consensus_queue.cc:237] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.610424 1176 tablet_bootstrap.cc:492] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.610709 1176 ts_tablet_manager.cc:1403] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.011s
I20260505 07:54:03.612039 1176 raft_consensus.cc:359] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.612326 1176 raft_consensus.cc:385] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.612442 1176 raft_consensus.cc:740] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.612735 1176 consensus_queue.cc:260] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.613909 1176 ts_tablet_manager.cc:1434] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:03.614428 1176 tablet_bootstrap.cc:492] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.617647 1176 tablet_bootstrap.cc:654] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.620280 1864 consensus_queue.cc:1048] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.621165 1864 raft_consensus.cc:493] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.621290 1864 raft_consensus.cc:515] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.621834 1864 leader_election.cc:290] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.621943 1848 consensus_queue.cc:1048] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.622565 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "067c32ecccb44bca8bf9cc3f6c229326" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:03.623026 717 leader_election.cc:343] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.624774 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "067c32ecccb44bca8bf9cc3f6c229326" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.625073 917 raft_consensus.cc:2468] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.625626 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d1ce00e487d4702a6affcad57a7fbe1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.625993 781 raft_consensus.cc:2468] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:03.627533 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d1ce00e487d4702a6affcad57a7fbe1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:03.627825 778 raft_consensus.cc:3060] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.629695 1848 consensus_queue.cc:1048] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.631335 778 raft_consensus.cc:2468] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:03.632452 718 leader_election.cc:304] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.633440 1596 leader_election.cc:290] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.636065 1645 raft_consensus.cc:2804] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.636253 1645 raft_consensus.cc:493] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.636574 1645 raft_consensus.cc:3060] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.644073 920 raft_consensus.cc:2468] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:03.645501 991 leader_election.cc:304] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.645848 1879 raft_consensus.cc:2804] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.645996 1879 raft_consensus.cc:697] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:03.646415 1879 consensus_queue.cc:237] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.655354 1172 tablet_bootstrap.cc:492] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.655534 1172 ts_tablet_manager.cc:1403] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.344s user 0.005s sys 0.021s
I20260505 07:54:03.656507 1172 raft_consensus.cc:359] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.656738 1172 raft_consensus.cc:385] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.656816 1172 raft_consensus.cc:740] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.657066 1172 consensus_queue.cc:260] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.657678 1172 ts_tablet_manager.cc:1434] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:03.657989 1172 tablet_bootstrap.cc:492] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.658409 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_38d26c09769b42159b828ec5bb429075 [id=fc9aba6852ee42cc83697f86e0e69c5e]
I20260505 07:54:03.658948 1684 catalog_manager.cc:679] Loaded metadata for table default.Table_28c935edd66e47959a2df29d39dfe13c [id=fe9a0701ecd44cd28915885958b3b3c6]
I20260505 07:54:03.659916 1172 tablet_bootstrap.cc:654] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.665294 1848 consensus_queue.cc:1048] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.670027 1866 consensus_queue.cc:1048] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.680249 1172 tablet_bootstrap.cc:492] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.680431 1172 ts_tablet_manager.cc:1403] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.007s
I20260505 07:54:03.681468 1172 raft_consensus.cc:359] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.681696 1172 raft_consensus.cc:385] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.681814 1172 raft_consensus.cc:740] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.682124 1176 tablet_bootstrap.cc:492] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.682405 1176 ts_tablet_manager.cc:1403] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.068s user 0.006s sys 0.005s
I20260505 07:54:03.686149 1176 raft_consensus.cc:359] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.686489 1176 raft_consensus.cc:385] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.686648 1176 raft_consensus.cc:740] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.687043 1176 consensus_queue.cc:260] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.688231 1176 ts_tablet_manager.cc:1434] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260505 07:54:03.688822 1176 tablet_bootstrap.cc:492] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.691236 1172 consensus_queue.cc:260] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.693778 1172 ts_tablet_manager.cc:1434] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.013s user 0.003s sys 0.000s
I20260505 07:54:03.695370 1176 tablet_bootstrap.cc:654] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.695678 1172 tablet_bootstrap.cc:492] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.697633 1172 tablet_bootstrap.cc:654] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.707348 1176 tablet_bootstrap.cc:492] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.707840 1176 ts_tablet_manager.cc:1403] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.008s
I20260505 07:54:03.709887 1176 raft_consensus.cc:359] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.710207 1176 raft_consensus.cc:385] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.710309 1176 raft_consensus.cc:740] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.710757 1176 consensus_queue.cc:260] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.711182 1512 raft_consensus.cc:493] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.711421 1512 raft_consensus.cc:515] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.711980 1176 ts_tablet_manager.cc:1434] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:03.712594 1176 tablet_bootstrap.cc:492] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.713732 1172 tablet_bootstrap.cc:492] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.713621 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c822fafe72544ca99fde910c8e764755" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.713999 1172 ts_tablet_manager.cc:1403] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.002s
W20260505 07:54:03.714370 717 leader_election.cc:343] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.714741 1172 raft_consensus.cc:359] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.714931 1172 raft_consensus.cc:385] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.715016 1172 raft_consensus.cc:740] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.715086 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c822fafe72544ca99fde910c8e764755" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.715242 1172 consensus_queue.cc:260] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.715484 917 raft_consensus.cc:2468] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.715766 1172 ts_tablet_manager.cc:1434] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:03.716326 718 leader_election.cc:304] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.716635 1864 raft_consensus.cc:2804] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.716869 1864 raft_consensus.cc:493] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.716926 1512 leader_election.cc:290] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:03.716980 1864 raft_consensus.cc:3060] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.717940 1172 tablet_bootstrap.cc:492] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.719499 1172 tablet_bootstrap.cc:654] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.719902 1864 raft_consensus.cc:515] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.720897 1864 leader_election.cc:290] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:03.720913 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c822fafe72544ca99fde910c8e764755" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.721249 917 raft_consensus.cc:3060] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.721338 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c822fafe72544ca99fde910c8e764755" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:03.721978 717 leader_election.cc:343] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.722947 1176 tablet_bootstrap.cc:654] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.724943 917 raft_consensus.cc:2468] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:03.725488 718 leader_election.cc:304] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.725875 1864 raft_consensus.cc:2804] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.726013 1864 raft_consensus.cc:697] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:03.726277 1864 consensus_queue.cc:237] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.729311 1569 consensus_queue.cc:1048] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.732784 1181 tablet_bootstrap.cc:492] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:03.732990 1181 ts_tablet_manager.cc:1403] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.357s user 0.056s sys 0.125s
I20260505 07:54:03.732625 1569 raft_consensus.cc:493] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.733901 1181 raft_consensus.cc:359] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.734141 1181 raft_consensus.cc:385] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.734223 1181 raft_consensus.cc:740] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.733764 1569 raft_consensus.cc:515] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.738380 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba2cc218300472cb5fd6dc8cfb7811c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:03.739017 856 leader_election.cc:343] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.738325 1181 consensus_queue.cc:260] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.740986 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba2cc218300472cb5fd6dc8cfb7811c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:03.741564 854 leader_election.cc:343] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.741685 854 leader_election.cc:304] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.742161 1848 raft_consensus.cc:2749] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.742786 1181 ts_tablet_manager.cc:1434] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260505 07:54:03.742520 1569 leader_election.cc:290] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:03.744264 1181 tablet_bootstrap.cc:492] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:03.747429 1181 tablet_bootstrap.cc:654] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.747927 1172 tablet_bootstrap.cc:492] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.748080 1172 ts_tablet_manager.cc:1403] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20260505 07:54:03.748934 1172 raft_consensus.cc:359] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.749140 1172 raft_consensus.cc:385] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.749243 1172 raft_consensus.cc:740] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.749480 1172 consensus_queue.cc:260] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.750099 1172 ts_tablet_manager.cc:1434] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:03.750403 1172 tablet_bootstrap.cc:492] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.752521 1172 tablet_bootstrap.cc:654] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.761909 1172 tablet_bootstrap.cc:492] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.762140 1172 ts_tablet_manager.cc:1403] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.002s
I20260505 07:54:03.762478 1569 raft_consensus.cc:493] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.762878 1569 raft_consensus.cc:515] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.763106 1172 raft_consensus.cc:359] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.763485 1172 raft_consensus.cc:385] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.763722 1172 raft_consensus.cc:740] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.764205 1172 consensus_queue.cc:260] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.764753 1645 raft_consensus.cc:515] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.765437 1172 ts_tablet_manager.cc:1434] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:03.766108 1172 tablet_bootstrap.cc:492] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.766173 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "067c32ecccb44bca8bf9cc3f6c229326" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:03.766826 1880 raft_consensus.cc:493] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:03.766825 717 leader_election.cc:343] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.766963 1880 raft_consensus.cc:515] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.767685 1645 leader_election.cc:290] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.767676 1880 leader_election.cc:290] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.768025 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "067c32ecccb44bca8bf9cc3f6c229326" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:03.768260 920 raft_consensus.cc:3060] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.768469 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d4bc3154704ba2bfde83bca10f812d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.768728 778 raft_consensus.cc:2468] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:03.768803 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d4bc3154704ba2bfde83bca10f812d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.768828 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf76e868bbcb44b0b1887c84dd27f093" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.769052 777 raft_consensus.cc:2468] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:03.769265 856 leader_election.cc:343] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.769527 854 leader_election.cc:304] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.769729 988 leader_election.cc:304] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.769861 1848 raft_consensus.cc:2804] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.770004 1880 raft_consensus.cc:2804] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.770112 1880 raft_consensus.cc:493] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.770202 1880 raft_consensus.cc:3060] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.770892 1864 raft_consensus.cc:493] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.771157 1864 raft_consensus.cc:515] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.772259 1864 leader_election.cc:290] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.772423 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e62b7af9b847b7a9005e4345905c0d" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:03.772917 717 leader_election.cc:343] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.773231 1880 raft_consensus.cc:515] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.774835 1880 leader_election.cc:290] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.775607 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf76e868bbcb44b0b1887c84dd27f093" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:03.775813 777 raft_consensus.cc:3060] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.777062 1880 consensus_queue.cc:1048] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.777475 1864 raft_consensus.cc:493] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.777802 1864 raft_consensus.cc:515] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.779001 1864 leader_election.cc:290] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.779307 777 raft_consensus.cc:2468] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:03.779276 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4783dd0b2fbe40219852d850fc0d9a82" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.779716 988 leader_election.cc:304] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.781512 1804 raft_consensus.cc:2804] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.781740 1804 raft_consensus.cc:697] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:03.782148 1804 consensus_queue.cc:237] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.788152 1181 tablet_bootstrap.cc:492] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:03.790755 1181 ts_tablet_manager.cc:1403] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.004s
I20260505 07:54:03.792330 1181 raft_consensus.cc:359] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.796419 1181 raft_consensus.cc:385] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.797049 1181 raft_consensus.cc:740] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.797797 1181 consensus_queue.cc:260] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.801328 1181 ts_tablet_manager.cc:1434] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.010s user 0.000s sys 0.001s
I20260505 07:54:03.802867 1181 tablet_bootstrap.cc:492] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:03.806453 1181 tablet_bootstrap.cc:654] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.829514 1181 tablet_bootstrap.cc:492] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:03.831718 1181 ts_tablet_manager.cc:1403] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.003s
I20260505 07:54:03.833475 1181 raft_consensus.cc:359] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.835173 1870 consensus_queue.cc:1048] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.836489 1181 raft_consensus.cc:385] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.836607 1181 raft_consensus.cc:740] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.836818 1181 consensus_queue.cc:260] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.837406 1181 ts_tablet_manager.cc:1434] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260505 07:54:03.837741 1181 tablet_bootstrap.cc:492] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:03.839794 1181 tablet_bootstrap.cc:654] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.844955 1512 raft_consensus.cc:493] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.845220 1512 raft_consensus.cc:515] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.846464 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003cd4472b424aacbba8c5f89900e228" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.846676 1512 leader_election.cc:290] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.846722 1050 raft_consensus.cc:2468] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.847076 717 leader_election.cc:304] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:03.847555 1512 raft_consensus.cc:2804] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.847746 1512 raft_consensus.cc:493] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.847931 1512 raft_consensus.cc:3060] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.848381 1569 leader_election.cc:290] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:03.848721 1569 raft_consensus.cc:493] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.848889 1569 raft_consensus.cc:515] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:03.849591 1569 leader_election.cc:290] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:03.849781 1569 raft_consensus.cc:493] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.849926 1569 raft_consensus.cc:515] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.850600 1569 leader_election.cc:290] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:03.851977 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42360f8ee4ff47c2af8d287f792207de" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.852864 1848 raft_consensus.cc:493] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260505 07:54:03.853231 856 leader_election.cc:343] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.854343 1848 raft_consensus.cc:3060] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.853744 915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e62b7af9b847b7a9005e4345905c0d" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.854918 915 raft_consensus.cc:2468] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.856493 1172 tablet_bootstrap.cc:654] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.863085 1848 raft_consensus.cc:515] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.864701 1848 leader_election.cc:290] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:03.865602 1512 raft_consensus.cc:515] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.866253 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d4bc3154704ba2bfde83bca10f812d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:03.866956 856 leader_election.cc:343] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.869091 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d11a9975854940a0be1693508f0841" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:03.869904 856 leader_election.cc:343] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.873785 920 raft_consensus.cc:2468] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:03.903785 854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 72254.6us
I20260505 07:54:03.905057 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4783dd0b2fbe40219852d850fc0d9a82" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.908789 1580 raft_consensus.cc:493] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.909104 1580 raft_consensus.cc:515] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.910153 1879 raft_consensus.cc:493] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.910307 1879 raft_consensus.cc:515] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.910902 1879 leader_election.cc:290] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.911407 1580 leader_election.cc:290] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.908232 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003cd4472b424aacbba8c5f89900e228" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.919349 921 raft_consensus.cc:2393] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:03.919701 919 raft_consensus.cc:2468] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:03.919994 1172 tablet_bootstrap.cc:492] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:03.920542 1172 ts_tablet_manager.cc:1403] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.155s user 0.008s sys 0.006s
I20260505 07:54:03.905611 718 leader_election.cc:304] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.925634 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa401bb4634022bca3c6b23421e4d5" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.925725 718 leader_election.cc:304] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.926147 718 leader_election.cc:304] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.927842 1779 raft_consensus.cc:2804] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.927970 1779 raft_consensus.cc:493] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.928121 1779 raft_consensus.cc:3060] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.930953 1779 raft_consensus.cc:515] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.932629 1779 leader_election.cc:290] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.932871 1779 raft_consensus.cc:2804] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.933077 1779 raft_consensus.cc:697] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:03.933405 1779 consensus_queue.cc:237] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.934870 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "067c32ecccb44bca8bf9cc3f6c229326" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.906502 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d11a9975854940a0be1693508f0841" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.935559 781 raft_consensus.cc:2468] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:03.907792 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d4bc3154704ba2bfde83bca10f812d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:03.936067 782 raft_consensus.cc:3060] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.933120 1864 raft_consensus.cc:3060] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.939160 782 raft_consensus.cc:2468] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:03.908572 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42360f8ee4ff47c2af8d287f792207de" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.940313 1864 raft_consensus.cc:2749] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:03.942888 991 leader_election.cc:343] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: bf76e868bbcb44b0b1887c84dd27f093
I20260505 07:54:03.943049 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e62b7af9b847b7a9005e4345905c0d" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
W20260505 07:54:03.943270 991 leader_election.cc:343] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: bf76e868bbcb44b0b1887c84dd27f093
I20260505 07:54:03.943464 919 raft_consensus.cc:3060] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.944669 1879 consensus_queue.cc:1048] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.940800 854 leader_election.cc:304] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.945139 1879 consensus_queue.cc:1048] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:03.945669 718 consensus_peers.cc:597] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7585f00119e14006990ed551667b4486. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:03.946349 1879 consensus_queue.cc:1048] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.944406 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "067c32ecccb44bca8bf9cc3f6c229326" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.947237 916 raft_consensus.cc:2393] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:03.948254 991 leader_election.cc:304] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.948974 1804 raft_consensus.cc:3060] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.949258 854 leader_election.cc:304] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.950305 1866 raft_consensus.cc:2804] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:03.950423 1866 raft_consensus.cc:493] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:03.952533 1804 raft_consensus.cc:2749] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.954897 1866 raft_consensus.cc:3060] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.955364 1512 leader_election.cc:290] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.956331 1172 raft_consensus.cc:359] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.956501 1172 raft_consensus.cc:385] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.956888 1172 raft_consensus.cc:740] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.957103 1172 consensus_queue.cc:260] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.958246 1172 ts_tablet_manager.cc:1434] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.037s user 0.001s sys 0.001s
I20260505 07:54:03.959965 1172 tablet_bootstrap.cc:492] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:03.960202 1512 raft_consensus.cc:493] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.960510 1512 raft_consensus.cc:515] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.965167 1684 tablet_loader.cc:96] loaded metadata for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640])
I20260505 07:54:03.962837 1176 tablet_bootstrap.cc:492] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:03.967306 1176 ts_tablet_manager.cc:1403] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.255s user 0.013s sys 0.049s
I20260505 07:54:03.968369 1512 leader_election.cc:290] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:03.969038 1176 raft_consensus.cc:359] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.961051 1906 raft_consensus.cc:493] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:03.969594 1176 raft_consensus.cc:385] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:03.969957 1176 raft_consensus.cc:740] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:03.970672 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "23fa401bb4634022bca3c6b23421e4d5" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.970412 1176 consensus_queue.cc:260] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.971117 1909 raft_consensus.cc:2804] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:03.971472 1909 raft_consensus.cc:697] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
W20260505 07:54:03.972448 854 leader_election.cc:343] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.973409 921 raft_consensus.cc:2393] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:03.973421 1176 ts_tablet_manager.cc:1434] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260505 07:54:03.973922 1864 consensus_queue.cc:1048] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.974406 854 leader_election.cc:304] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.972045 1909 consensus_queue.cc:237] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.976382 1176 tablet_bootstrap.cc:492] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:03.977404 1848 raft_consensus.cc:2749] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:03.977591 1172 tablet_bootstrap.cc:654] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.979723 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397d33e908af4a0bb8dde26d273be01a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:03.980012 916 raft_consensus.cc:2393] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:03.980741 1866 raft_consensus.cc:515] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.981300 1176 tablet_bootstrap.cc:654] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:03.974975 1906 raft_consensus.cc:515] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:03.993422 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12501fdf2cc5491cb1a06db490e70810" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:03.952772 1596 consensus_queue.cc:1048] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:03.966609 1181 tablet_bootstrap.cc:492] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:03.970142 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "397d33e908af4a0bb8dde26d273be01a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:03.998871 1181 ts_tablet_manager.cc:1403] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.161s user 0.003s sys 0.011s
I20260505 07:54:03.976073 991 leader_election.cc:304] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:03.999166 717 leader_election.cc:304] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:03.999447 1864 raft_consensus.cc:3060] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.978863 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003cd4472b424aacbba8c5f89900e228" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:03.979401 1051 raft_consensus.cc:3060] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:03.981914 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d11a9975854940a0be1693508f0841" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:04.000363 1055 raft_consensus.cc:3060] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.001945 1864 raft_consensus.cc:2749] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:04.002287 856 leader_election.cc:343] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.985153 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e62b7af9b847b7a9005e4345905c0d" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:04.003288 1864 consensus_queue.cc:1048] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:04.003635 717 leader_election.cc:343] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:03.989146 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12501fdf2cc5491cb1a06db490e70810" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.006102 1864 raft_consensus.cc:493] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.006265 1864 raft_consensus.cc:515] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.006942 1864 leader_election.cc:290] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:04.007400 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9494c3b6a99487596a030856b371b68" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:04.009958 854 leader_election.cc:343] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.010529 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d11a9975854940a0be1693508f0841" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:04.010776 779 raft_consensus.cc:3060] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.010900 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9494c3b6a99487596a030856b371b68" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.010876 1906 leader_election.cc:290] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:04.013320 1176 tablet_bootstrap.cc:492] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.013615 779 raft_consensus.cc:2468] T 55d11a9975854940a0be1693508f0841 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
W20260505 07:54:04.013774 856 leader_election.cc:343] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.014127 1176 ts_tablet_manager.cc:1403] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.038s user 0.008s sys 0.002s
I20260505 07:54:04.014231 856 leader_election.cc:304] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:04.014741 854 leader_election.cc:304] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.015022 1848 raft_consensus.cc:2749] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.015226 1848 raft_consensus.cc:2804] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.015354 1848 raft_consensus.cc:697] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:04.015719 1181 raft_consensus.cc:359] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.015834 1848 consensus_queue.cc:237] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.016851 1866 leader_election.cc:290] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:04.009142 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "003cd4472b424aacbba8c5f89900e228" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:04.020685 921 raft_consensus.cc:3060] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.020540 1176 raft_consensus.cc:359] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.021519 1176 raft_consensus.cc:385] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.022092 1176 raft_consensus.cc:740] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.023319 1176 consensus_queue.cc:260] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:04.025491 718 consensus_peers.cc:597] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f92c53c68a4b4e319e98b84128e8e6a4. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:04.027640 1176 ts_tablet_manager.cc:1434] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260505 07:54:04.033515 1176 tablet_bootstrap.cc:492] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.034153 1779 consensus_queue.cc:1048] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:04.018790 988 consensus_peers.cc:597] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:04.016000 1181 raft_consensus.cc:385] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.040858 1181 raft_consensus.cc:740] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.023555 1596 raft_consensus.cc:2749] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.028674 1055 raft_consensus.cc:2468] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.047859 1181 consensus_queue.cc:260] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.048164 1053 raft_consensus.cc:2468] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:04.051687 717 leader_election.cc:304] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:04.052220 717 leader_election.cc:304] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:04.052337 917 raft_consensus.cc:2468] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:04.053215 1181 ts_tablet_manager.cc:1434] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.054s user 0.001s sys 0.001s
I20260505 07:54:04.053678 1181 tablet_bootstrap.cc:492] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.056214 1181 tablet_bootstrap.cc:654] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.056689 919 raft_consensus.cc:2468] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.060168 921 raft_consensus.cc:2468] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.061844 1172 tablet_bootstrap.cc:492] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.062188 1172 ts_tablet_manager.cc:1403] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.102s user 0.021s sys 0.017s
I20260505 07:54:04.063973 1172 raft_consensus.cc:359] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.064459 1172 raft_consensus.cc:385] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.064631 1172 raft_consensus.cc:740] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.066181 1172 consensus_queue.cc:260] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:04.067865 445 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.61:33565, user_credentials={real_user=slave}} blocked reactor thread for 171984us
I20260505 07:54:04.068907 1172 ts_tablet_manager.cc:1434] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260505 07:54:04.069861 1172 tablet_bootstrap.cc:492] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.074952 1172 tablet_bootstrap.cc:654] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.075814 1909 raft_consensus.cc:493] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.076160 1909 raft_consensus.cc:515] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.077968 1909 leader_election.cc:290] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:04.078853 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab3850799254dda9716fd5bf9404159" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:04.079536 856 leader_election.cc:343] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.098884 1684 tablet_loader.cc:96] loaded metadata for tablet 02716f62aa354d8f839266661b7f05ea (table default.Table_a86abb495238445381ac0202804b4ca0 [id=4d3b183ab74a4078beb7583c9e60e2c6])
I20260505 07:54:04.113957 1878 consensus_queue.cc:1048] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.118842 1779 raft_consensus.cc:2804] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.120497 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab3850799254dda9716fd5bf9404159" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:04.121109 1512 raft_consensus.cc:2804] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.121480 718 leader_election.cc:304] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.124213 1649 raft_consensus.cc:493] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.124272 1684 tablet_loader.cc:96] loaded metadata for tablet 036d306d592f41cb96814aa60e30551e (table default.Table_911918ab5f5145178d2490fb80e7db21 [id=218966c8e4af4210a14adcb68269cda4])
I20260505 07:54:04.125141 1870 raft_consensus.cc:2804] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.125319 1649 raft_consensus.cc:515] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.124995 1512 raft_consensus.cc:697] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:04.133910 1779 raft_consensus.cc:493] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.135530 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accb2dedc30e494ebd22c33594cee661" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.135659 1649 leader_election.cc:290] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.135735 1176 tablet_bootstrap.cc:654] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.140087 1779 raft_consensus.cc:3060] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.155889 1684 tablet_loader.cc:96] loaded metadata for tablet 05351965bdba4ccca8d13223cd505948 (table default.Table_54f371d41b5e4a54b08bb82a834ac0af [id=abda2c3e5e9d4c64b0d08510cb3d8aed])
I20260505 07:54:04.159209 1512 consensus_queue.cc:237] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:04.169165 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 47183.4us
I20260505 07:54:04.169697 1804 consensus_queue.cc:1048] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:04.171545 854 leader_election.cc:343] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.171729 854 leader_election.cc:304] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:04.172178 1848 raft_consensus.cc:2749] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.173480 1870 raft_consensus.cc:697] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:04.173364 1779 raft_consensus.cc:515] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.174002 1870 consensus_queue.cc:237] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.176484 1684 tablet_loader.cc:96] loaded metadata for tablet 08c6ed755fc64080a2310fe9a4e0a68b (table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b])
I20260505 07:54:04.176702 1804 consensus_queue.cc:1048] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.177266 1684 tablet_loader.cc:96] loaded metadata for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57])
I20260505 07:54:04.177954 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "accb2dedc30e494ebd22c33594cee661" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.178190 920 raft_consensus.cc:2393] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:04.179888 1779 leader_election.cc:290] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:04.182579 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9494c3b6a99487596a030856b371b68" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:04.182885 1049 raft_consensus.cc:3060] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.186609 1049 raft_consensus.cc:2468] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.201700 1176 tablet_bootstrap.cc:492] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.201893 1176 ts_tablet_manager.cc:1403] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.169s user 0.013s sys 0.000s
I20260505 07:54:04.202672 1176 raft_consensus.cc:359] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.202857 1176 raft_consensus.cc:385] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.202940 1176 raft_consensus.cc:740] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.203126 1176 consensus_queue.cc:260] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.203683 1176 ts_tablet_manager.cc:1434] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:04.203974 1176 tablet_bootstrap.cc:492] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.205664 1176 tablet_bootstrap.cc:654] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.209029 1684 tablet_loader.cc:96] loaded metadata for tablet 0bd2e2514c72419390c1bb9bd13f1fb3 (table default.Table_f75ef4b5e6ae433f8c6bee72fc30eaf2 [id=f35c58e2d1f54d3ba63a303a0648cebf])
I20260505 07:54:04.210008 1684 tablet_loader.cc:96] loaded metadata for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e])
I20260505 07:54:04.210651 1684 tablet_loader.cc:96] loaded metadata for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09])
I20260505 07:54:04.211416 1684 tablet_loader.cc:96] loaded metadata for tablet 0e9a98243da14d43bc649bb19aed4f0c (table default.Table_21673305a26a4342a0be11d98e875447 [id=76c4943b63be49fdaf6ee51abe09b9b3])
I20260505 07:54:04.214395 1176 tablet_bootstrap.cc:492] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.214551 1176 ts_tablet_manager.cc:1403] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260505 07:54:04.215308 1176 raft_consensus.cc:359] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.215538 1176 raft_consensus.cc:385] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.215607 1932 raft_consensus.cc:493] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.215667 1176 raft_consensus.cc:740] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.215747 1932 raft_consensus.cc:515] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.215881 1176 consensus_queue.cc:260] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.216414 1932 leader_election.cc:290] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.216526 1176 ts_tablet_manager.cc:1434] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:04.216821 1176 tablet_bootstrap.cc:492] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.218585 1176 tablet_bootstrap.cc:654] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.191958 1181 tablet_bootstrap.cc:492] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:04.217217 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9044763bed1a41739ecce4def4cca1fe" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.219858 1053 raft_consensus.cc:2468] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:04.220279 1181 ts_tablet_manager.cc:1403] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.167s user 0.005s sys 0.000s
I20260505 07:54:04.220367 717 leader_election.cc:304] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:04.220943 1932 raft_consensus.cc:2804] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.221045 1932 raft_consensus.cc:493] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.221113 1932 raft_consensus.cc:3060] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.221127 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9494c3b6a99487596a030856b371b68" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:04.221138 1181 raft_consensus.cc:359] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.221308 1181 raft_consensus.cc:385] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.221310 920 raft_consensus.cc:3060] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.221377 1181 raft_consensus.cc:740] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.221621 1181 consensus_queue.cc:260] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.222087 1181 ts_tablet_manager.cc:1434] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:04.223982 920 raft_consensus.cc:2468] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.224782 1932 raft_consensus.cc:515] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.225869 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9044763bed1a41739ecce4def4cca1fe" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.226109 920 raft_consensus.cc:2468] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:04.226748 1932 leader_election.cc:290] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.227520 718 leader_election.cc:304] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.227792 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9044763bed1a41739ecce4def4cca1fe" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:04.227983 1053 raft_consensus.cc:3060] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.231086 1053 raft_consensus.cc:2468] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.236847 1181 tablet_bootstrap.cc:492] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.238492 1181 tablet_bootstrap.cc:654] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.240422 1934 raft_consensus.cc:3060] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.244274 1172 tablet_bootstrap.cc:492] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.244702 1172 ts_tablet_manager.cc:1403] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.175s user 0.022s sys 0.028s
I20260505 07:54:04.247458 1172 raft_consensus.cc:359] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.248061 1172 raft_consensus.cc:385] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.248299 1172 raft_consensus.cc:740] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.249131 1172 consensus_queue.cc:260] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.251272 1172 ts_tablet_manager.cc:1434] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260505 07:54:04.252468 1172 tablet_bootstrap.cc:492] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.259141 1172 tablet_bootstrap.cc:654] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.259370 717 leader_election.cc:304] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:04.259436 1932 consensus_queue.cc:1048] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.027s
I20260505 07:54:04.261361 1684 tablet_loader.cc:96] loaded metadata for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3])
I20260505 07:54:04.262794 1684 tablet_loader.cc:96] loaded metadata for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_1f4b1af520d64a108ff4289226a1561d [id=a238daba389c412ea67318d99cfb5559])
I20260505 07:54:04.263897 1684 tablet_loader.cc:96] loaded metadata for tablet 1506cd31fe364e328867bbf70e36845e (table default.Table_ad8e6e942c73439fb64536b3ab1edc9c [id=c7cad0401e40485e9ed03160b9bed704])
I20260505 07:54:04.266556 1172 tablet_bootstrap.cc:492] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.266898 1172 ts_tablet_manager.cc:1403] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.006s
I20260505 07:54:04.267385 1181 tablet_bootstrap.cc:492] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:04.267619 1181 ts_tablet_manager.cc:1403] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.031s user 0.004s sys 0.000s
I20260505 07:54:04.268501 1172 raft_consensus.cc:359] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.268913 1172 raft_consensus.cc:385] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.268771 1181 raft_consensus.cc:359] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.269063 1172 raft_consensus.cc:740] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.269234 1181 raft_consensus.cc:385] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.269507 1181 raft_consensus.cc:740] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.269511 1172 consensus_queue.cc:260] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.270027 1181 consensus_queue.cc:260] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.270682 1172 ts_tablet_manager.cc:1434] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260505 07:54:04.271235 1172 tablet_bootstrap.cc:492] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.275079 1172 tablet_bootstrap.cc:654] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.276888 1181 ts_tablet_manager.cc:1434] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
W20260505 07:54:04.277505 1818 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260505 07:54:04.278053 1818 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260505 07:54:04.278239 1818 flags.cc:432] Enabled unsafe flag: --never_fsync=true
I20260505 07:54:04.280388 1181 tablet_bootstrap.cc:492] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.283422 1684 tablet_loader.cc:96] loaded metadata for tablet 172ca66c49354f549fa55fe63abf9970 (table default.Table_8c264d1fb97d491bb94ba720eef63e01 [id=4ca8dc50d71c42c6adbdedeb932d3040])
I20260505 07:54:04.286773 1172 tablet_bootstrap.cc:492] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.287014 1172 ts_tablet_manager.cc:1403] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.005s
I20260505 07:54:04.288472 1172 raft_consensus.cc:359] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.288866 1172 raft_consensus.cc:385] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.289023 1172 raft_consensus.cc:740] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.289422 1172 consensus_queue.cc:260] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:04.289841 718 consensus_peers.cc:597] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1ef4f516e64a44fd8cd33674fb808067. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:04.290136 717 leader_election.cc:304] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:04.292551 1932 raft_consensus.cc:2804] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.292961 1779 raft_consensus.cc:2804] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
W20260505 07:54:04.293678 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 60321.5us
W20260505 07:54:04.294432 1818 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260505 07:54:04.294531 1818 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260505 07:54:04.294632 1818 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260505 07:54:04.294593 1933 raft_consensus.cc:493] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:04.294695 1818 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260505 07:54:04.294642 1864 consensus_queue.cc:1048] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260505 07:54:04.294788 1933 raft_consensus.cc:515] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.295061 1932 raft_consensus.cc:697] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:04.295302 1932 consensus_queue.cc:237] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.295463 1933 leader_election.cc:290] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.296299 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9044763bed1a41739ecce4def4cca1fe" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:04.296515 920 raft_consensus.cc:3060] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.299189 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac5064f68d24ebe95b5a0b8e10f463e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.301137 1580 consensus_queue.cc:1048] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.301604 920 raft_consensus.cc:2468] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.302770 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac5064f68d24ebe95b5a0b8e10f463e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.304183 1779 raft_consensus.cc:697] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:04.306448 1864 raft_consensus.cc:493] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.308094 1933 consensus_queue.cc:1048] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260505 07:54:04.307927 1931 raft_consensus.cc:493] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.308868 1864 raft_consensus.cc:515] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.312789 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06904a54b1c5463da31af0bc00fb606f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.312793 1864 leader_election.cc:290] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.312788 1779 consensus_queue.cc:237] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.309072 1931 raft_consensus.cc:515] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:04.315217 717 leader_election.cc:343] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.315621 718 leader_election.cc:304] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.317625 1864 raft_consensus.cc:2749] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.319983 1172 ts_tablet_manager.cc:1434] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.033s user 0.001s sys 0.002s
I20260505 07:54:04.320592 1172 tablet_bootstrap.cc:492] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.321924 1934 raft_consensus.cc:2749] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.323473 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46120e7a5dcb482f8f93fc2aa5219aff" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.323298 1931 leader_election.cc:290] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:04.324194 717 leader_election.cc:343] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.330925 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06904a54b1c5463da31af0bc00fb606f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.330919 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46120e7a5dcb482f8f93fc2aa5219aff" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.331205 916 raft_consensus.cc:2468] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:04.331215 913 raft_consensus.cc:2468] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:04.331944 718 leader_election.cc:304] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.332209 1931 raft_consensus.cc:2804] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.332525 1931 raft_consensus.cc:493] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.332666 1931 raft_consensus.cc:3060] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.333695 1684 tablet_loader.cc:96] loaded metadata for tablet 1776a12afc434d639cf98a9d253ef2d6 (table default.Table_4805d11726a3446b86b7a990c0dbb3c5 [id=035a76f8e6f34bc1bb1c75c4908bb4d7])
I20260505 07:54:04.336400 718 leader_election.cc:304] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.337394 1804 consensus_queue.cc:1048] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.337580 1176 tablet_bootstrap.cc:492] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.338614 1176 ts_tablet_manager.cc:1403] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.122s user 0.015s sys 0.000s
I20260505 07:54:04.339385 1181 tablet_bootstrap.cc:654] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.339794 1684 tablet_loader.cc:96] loaded metadata for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f])
I20260505 07:54:04.339618 1931 raft_consensus.cc:515] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.340804 1931 leader_election.cc:290] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.339746 1779 raft_consensus.cc:2804] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.341284 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06904a54b1c5463da31af0bc00fb606f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:04.341461 913 raft_consensus.cc:3060] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.341490 1779 raft_consensus.cc:493] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.341609 1779 raft_consensus.cc:3060] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.343760 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06904a54b1c5463da31af0bc00fb606f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:04.344228 717 leader_election.cc:343] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.344997 1684 tablet_loader.cc:96] loaded metadata for tablet 180157193a184add848ba67ca8042109 (table default.Table_6ddd9e0382d845d1b89edce006361e1a [id=ba1b026067294e0cae324fb994815400])
I20260505 07:54:04.348646 913 raft_consensus.cc:2468] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.349260 1934 consensus_queue.cc:1048] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.349319 718 leader_election.cc:304] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.350385 1934 raft_consensus.cc:2804] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.350932 1934 raft_consensus.cc:697] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:04.344545 1779 raft_consensus.cc:515] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.351366 1934 consensus_queue.cc:237] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.353513 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46120e7a5dcb482f8f93fc2aa5219aff" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:04.353753 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46120e7a5dcb482f8f93fc2aa5219aff" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:04.353996 916 raft_consensus.cc:3060] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:04.354008 717 leader_election.cc:343] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.356884 916 raft_consensus.cc:2468] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.357317 718 leader_election.cc:304] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.357487 1172 tablet_bootstrap.cc:654] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.357719 1779 leader_election.cc:290] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.357764 1931 raft_consensus.cc:2804] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.357970 1931 raft_consensus.cc:697] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:04.358227 1931 consensus_queue.cc:237] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.362576 1932 consensus_queue.cc:1048] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.360335 1176 raft_consensus.cc:359] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.364169 1176 raft_consensus.cc:385] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.364926 1176 raft_consensus.cc:740] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.366647 1176 consensus_queue.cc:260] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.369907 1176 ts_tablet_manager.cc:1434] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.029s user 0.003s sys 0.001s
I20260505 07:54:04.370630 1176 tablet_bootstrap.cc:492] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.373101 1804 consensus_queue.cc:1048] T 70404184502a4394a737518220f2124f P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.378922 1804 raft_consensus.cc:493] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.379060 1804 raft_consensus.cc:515] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.379712 1804 leader_election.cc:290] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.379791 1176 tablet_bootstrap.cc:654] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.380380 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b57c1c618484d5d86a846fb982a2240" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:04.380575 778 raft_consensus.cc:2468] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:04.380915 988 leader_election.cc:304] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:04.381590 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b57c1c618484d5d86a846fb982a2240" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.381635 1804 raft_consensus.cc:2804] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.381742 1804 raft_consensus.cc:493] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.381767 919 raft_consensus.cc:2468] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:04.381832 1804 raft_consensus.cc:3060] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.361850 1684 tablet_loader.cc:96] loaded metadata for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_ed8e2add9bbf436f812de7e3fe2ed692 [id=2a512c2695b540c2a9e14616ee257125])
I20260505 07:54:04.384126 1818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.105.20:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:35401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.0.105.60
--webserver_port=44877
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 0ba7f54e15c9f673bf3e350f30b8d5f1a4037b79
build type FASTDEBUG
built by None at 05 May 2026 07:43:22 UTC on bdcb31816ec0
build id 11754
ASAN enabled
I20260505 07:54:04.384838 1684 tablet_loader.cc:96] loaded metadata for tablet 2045202714c14aa388d1e8bc18e2063d (table default.Table_657cc0ba6f494479bd433c8c0279c734 [id=f191fbb56e234f10a9f94b2b998781bd])
I20260505 07:54:04.384896 1804 raft_consensus.cc:515] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.386091 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b57c1c618484d5d86a846fb982a2240" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:04.386286 778 raft_consensus.cc:3060] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.387144 1818 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260505 07:54:04.389923 1818 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20260505 07:54:04.391564 778 raft_consensus.cc:2468] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:04.391989 988 leader_election.cc:304] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:04.392288 1803 raft_consensus.cc:2804] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.392412 1803 raft_consensus.cc:697] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:04.392683 1803 consensus_queue.cc:237] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.394693 1053 raft_consensus.cc:3060] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.396703 1804 leader_election.cc:290] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.396715 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b57c1c618484d5d86a846fb982a2240" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:04.396900 919 raft_consensus.cc:3060] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.400501 919 raft_consensus.cc:2468] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:04.409893 1804 consensus_queue.cc:1048] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.414813 1172 tablet_bootstrap.cc:492] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.415184 1172 ts_tablet_manager.cc:1403] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.095s user 0.001s sys 0.009s
W20260505 07:54:04.415313 1969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:54:04.415624 1684 tablet_loader.cc:96] loaded metadata for tablet 212e496d564a4b5ebabb22504be76bf2 (table default.Table_7001b868854949a5b9bee0ba45c2f522 [id=978b4030e04a4e32938aa646b1e85e6c])
I20260505 07:54:04.417122 1172 raft_consensus.cc:359] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.417634 1172 raft_consensus.cc:385] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.417809 1172 raft_consensus.cc:740] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.418078 1684 tablet_loader.cc:96] loaded metadata for tablet 21480c42b73e4bf0ab2453b085a867c0 (table default.Table_db54454473d84a83b6c77d06a1f09618 [id=0faa6676bab04919984b580299143271])
I20260505 07:54:04.418392 1172 consensus_queue.cc:260] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.420492 1172 ts_tablet_manager.cc:1434] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260505 07:54:04.422259 1779 consensus_queue.cc:1048] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.423570 1906 consensus_queue.cc:1048] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.426133 1803 consensus_queue.cc:1048] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.426477 778 raft_consensus.cc:3060] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.433909 1181 tablet_bootstrap.cc:492] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:04.434103 1181 ts_tablet_manager.cc:1403] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.155s user 0.004s sys 0.000s
I20260505 07:54:04.434980 1181 raft_consensus.cc:359] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.435263 1181 raft_consensus.cc:385] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.435352 1181 raft_consensus.cc:740] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.435542 1176 tablet_bootstrap.cc:492] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.436064 1172 tablet_bootstrap.cc:492] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.447183 1176 ts_tablet_manager.cc:1403] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.067s user 0.000s sys 0.008s
I20260505 07:54:04.448042 1580 consensus_queue.cc:1048] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.445487 1181 consensus_queue.cc:260] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.448607 1181 ts_tablet_manager.cc:1434] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20260505 07:54:04.450949 1684 tablet_loader.cc:96] loaded metadata for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_1e6b1ec4673c45e6b7b6c2696722d84b [id=5d0d6b47936a454c9a93cc8af031807b])
I20260505 07:54:04.451247 1176 raft_consensus.cc:359] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.451802 1866 consensus_queue.cc:1048] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.452072 1176 raft_consensus.cc:385] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.452442 1176 raft_consensus.cc:740] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.453975 1176 consensus_queue.cc:260] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.455163 1172 tablet_bootstrap.cc:654] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.455364 1866 raft_consensus.cc:493] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.455495 1866 raft_consensus.cc:515] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.456472 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8c9eaaef250461ca7a37d0cf42b07ee" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.456615 1176 ts_tablet_manager.cc:1434] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.009s user 0.000s sys 0.003s
I20260505 07:54:04.457865 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8c9eaaef250461ca7a37d0cf42b07ee" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:04.457789 1176 tablet_bootstrap.cc:492] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.461009 1866 leader_election.cc:290] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:04.461624 856 leader_election.cc:343] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.463013 1596 consensus_queue.cc:1048] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:04.463724 854 leader_election.cc:343] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.464730 1176 tablet_bootstrap.cc:654] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:04.469105 1970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:54:04.476184 1181 tablet_bootstrap.cc:492] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.477841 1181 tablet_bootstrap.cc:654] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.483539 1684 tablet_loader.cc:96] loaded metadata for tablet 225c56ac920b448c89c38d6fb3c1bdef (table default.Table_ec95dd1d091844c3adb51935b2e48795 [id=8b6dda11eb884c7da54b226291f403c5])
I20260505 07:54:04.487887 854 leader_election.cc:304] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:04.488526 1866 raft_consensus.cc:2749] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.491178 1779 consensus_queue.cc:1048] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:04.499218 717 consensus_peers.cc:597] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:04.502378 1866 raft_consensus.cc:493] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.502552 1866 raft_consensus.cc:515] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.502795 1176 tablet_bootstrap.cc:492] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.503299 1866 leader_election.cc:290] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:04.503656 1176 ts_tablet_manager.cc:1403] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.046s user 0.007s sys 0.015s
I20260505 07:54:04.504144 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1feb0387bc4afead72346398dbe9e5" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.504303 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1feb0387bc4afead72346398dbe9e5" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:04.504772 856 leader_election.cc:343] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:04.505110 854 leader_election.cc:343] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.505239 854 leader_election.cc:304] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:04.505658 1866 raft_consensus.cc:2749] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.509653 1176 raft_consensus.cc:359] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.511514 1684 tablet_loader.cc:96] loaded metadata for tablet 24f4854e8752493fb832ae2c12a033ce (table default.Table_af7b8d3a4e6843348198094736404edb [id=1a1c2d94983e409c8b75400de8adbae8])
I20260505 07:54:04.511637 1176 raft_consensus.cc:385] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.512235 1684 tablet_loader.cc:96] loaded metadata for tablet 25072055f5534078adf5b5025614f951 (table default.Table_f93a81c5dfe740819f0f6ea5bf7f1067 [id=a8a6d42a6b6748d590ef5949a9d1e650])
I20260505 07:54:04.512315 1176 raft_consensus.cc:740] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.513957 1176 consensus_queue.cc:260] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.518904 1176 ts_tablet_manager.cc:1434] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.014s user 0.000s sys 0.005s
I20260505 07:54:04.521377 1176 tablet_bootstrap.cc:492] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.536194 1176 tablet_bootstrap.cc:654] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.540138 1684 tablet_loader.cc:96] loaded metadata for tablet 2706bc9b9e02414693062deff5cfe81b (table default.Table_0410f2bbd0504738ba4aa1f7412e3405 [id=617f784f561c4c9c93be1bdd500fc4d9])
I20260505 07:54:04.541095 1684 tablet_loader.cc:96] loaded metadata for tablet 27f00169aad9420b892100acb4256f92 (table default.Table_97ee3338c1344152a0d674d42c76d5be [id=908887d51e8949ac81cebbcb0c6ddb6d])
I20260505 07:54:04.541998 1684 tablet_loader.cc:96] loaded metadata for tablet 29a82ed2b87c487aac84194f2dcb0d6b (table default.Table_7b4336be586a4fc1a8dada3651641fd9 [id=057b807a46cb43b4a60dc61212a8b8a8])
I20260505 07:54:04.542438 1684 tablet_loader.cc:96] loaded metadata for tablet 2ca12a17fb014374bf7ed764a91e6985 (table default.Table_f3a7ead203124b49a437c53941ec4d58 [id=8ebfbe0aa09a43bdbb4b8f323658b3b1])
I20260505 07:54:04.544497 1181 tablet_bootstrap.cc:492] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:04.544713 1181 ts_tablet_manager.cc:1403] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.075s user 0.001s sys 0.002s
I20260505 07:54:04.545817 1181 raft_consensus.cc:359] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.546106 1181 raft_consensus.cc:385] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.546252 1181 raft_consensus.cc:740] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.546610 1181 consensus_queue.cc:260] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.553794 1176 tablet_bootstrap.cc:492] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.553973 1176 ts_tablet_manager.cc:1403] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.033s user 0.006s sys 0.004s
I20260505 07:54:04.554144 1818 server_base.cc:1061] running on GCE node
I20260505 07:54:04.554769 1176 raft_consensus.cc:359] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.554986 1176 raft_consensus.cc:385] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.555058 1176 raft_consensus.cc:740] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
W20260505 07:54:04.554914 1977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260505 07:54:04.555294 1176 consensus_queue.cc:260] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.555851 1176 ts_tablet_manager.cc:1434] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:04.556180 1176 tablet_bootstrap.cc:492] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.558072 1818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260505 07:54:04.561301 1176 tablet_bootstrap.cc:654] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.562002 1172 tablet_bootstrap.cc:492] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.562182 1172 ts_tablet_manager.cc:1403] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.136s user 0.008s sys 0.024s
I20260505 07:54:04.563042 1172 raft_consensus.cc:359] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.563242 1172 raft_consensus.cc:385] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.563345 1172 raft_consensus.cc:740] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.563578 1172 consensus_queue.cc:260] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.564249 1172 ts_tablet_manager.cc:1434] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:04.564610 1172 tablet_bootstrap.cc:492] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.566705 1172 tablet_bootstrap.cc:654] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.577019 1172 tablet_bootstrap.cc:492] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.577214 1172 ts_tablet_manager.cc:1403] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.007s
I20260505 07:54:04.578504 1172 raft_consensus.cc:359] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.578737 1596 consensus_queue.cc:1048] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.578732 1172 raft_consensus.cc:385] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.578925 1172 raft_consensus.cc:740] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.579164 1172 consensus_queue.cc:260] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.580624 1176 tablet_bootstrap.cc:492] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.580793 1176 ts_tablet_manager.cc:1403] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.005s
I20260505 07:54:04.581737 1176 raft_consensus.cc:359] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.581961 1176 raft_consensus.cc:385] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.582072 1176 raft_consensus.cc:740] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.582324 1176 consensus_queue.cc:260] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.582973 1176 ts_tablet_manager.cc:1434] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:04.583298 1176 tablet_bootstrap.cc:492] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.585383 1176 tablet_bootstrap.cc:654] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.586983 1172 ts_tablet_manager.cc:1434] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.010s user 0.001s sys 0.002s
I20260505 07:54:04.588475 1176 tablet_bootstrap.cc:492] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.588624 1176 ts_tablet_manager.cc:1403] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260505 07:54:04.589287 1818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260505 07:54:04.589857 1176 raft_consensus.cc:359] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.590092 1176 raft_consensus.cc:385] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.590184 1176 raft_consensus.cc:740] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.591897 1172 tablet_bootstrap.cc:492] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.593190 1818 hybrid_clock.cc:648] HybridClock initialized: now 1777967644593061 us; error 91 us; skew 500 ppm
I20260505 07:54:04.593807 1172 tablet_bootstrap.cc:654] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.594226 1818 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260505 07:54:04.594776 1176 consensus_queue.cc:260] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.595294 1176 ts_tablet_manager.cc:1434] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:54:04.595911 1176 tablet_bootstrap.cc:492] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.598692 1176 tablet_bootstrap.cc:654] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.602093 1181 ts_tablet_manager.cc:1434] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.057s user 0.001s sys 0.001s
I20260505 07:54:04.604264 1684 tablet_loader.cc:96] loaded metadata for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50])
I20260505 07:54:04.605806 1172 tablet_bootstrap.cc:492] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.605921 1684 tablet_loader.cc:96] loaded metadata for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_6de81e84d3e44828a5b8026b551bdeb3 [id=509c7b9d25aa48af9dde3825cfe313d0])
I20260505 07:54:04.605953 1172 ts_tablet_manager.cc:1403] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.008s
I20260505 07:54:04.606740 1172 raft_consensus.cc:359] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.606949 1172 raft_consensus.cc:385] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.607048 1172 raft_consensus.cc:740] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.607147 1181 tablet_bootstrap.cc:492] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.607265 1172 consensus_queue.cc:260] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.607836 1172 ts_tablet_manager.cc:1434] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:04.608129 1172 tablet_bootstrap.cc:492] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.609249 1181 tablet_bootstrap.cc:654] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.610096 1172 tablet_bootstrap.cc:654] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.611006 1878 consensus_queue.cc:1048] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.612221 1176 tablet_bootstrap.cc:492] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.612354 1176 ts_tablet_manager.cc:1403] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.002s
I20260505 07:54:04.613121 1176 raft_consensus.cc:359] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.613286 1176 raft_consensus.cc:385] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.613368 1176 raft_consensus.cc:740] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.613598 1176 consensus_queue.cc:260] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.614120 1176 ts_tablet_manager.cc:1434] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:04.614403 1176 tablet_bootstrap.cc:492] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.616017 1176 tablet_bootstrap.cc:654] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.616894 1866 raft_consensus.cc:493] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.617029 1866 raft_consensus.cc:515] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.617558 1866 leader_election.cc:290] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:04.618325 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde59d52fd234343b1ca13a7d4aae6bf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:04.619107 856 leader_election.cc:343] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.622073 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde59d52fd234343b1ca13a7d4aae6bf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:04.622400 776 raft_consensus.cc:2468] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:04.622812 854 leader_election.cc:304] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.623206 1866 raft_consensus.cc:2804] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.623306 1866 raft_consensus.cc:493] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.623404 1866 raft_consensus.cc:3060] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.626128 1866 raft_consensus.cc:515] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.627771 1866 leader_election.cc:290] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:04.628031 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde59d52fd234343b1ca13a7d4aae6bf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:04.628070 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde59d52fd234343b1ca13a7d4aae6bf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:04.628275 776 raft_consensus.cc:3060] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:04.628564 856 leader_election.cc:343] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.628769 1818 webserver.cc:492] Webserver started at http://127.0.105.60:44877/ using document root <none> and password file <none>
I20260505 07:54:04.631166 1818 fs_manager.cc:362] Metadata directory not provided
I20260505 07:54:04.631294 776 raft_consensus.cc:2468] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:04.631466 1818 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260505 07:54:04.631808 854 leader_election.cc:304] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.632177 1866 raft_consensus.cc:2804] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.632405 1866 raft_consensus.cc:697] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:04.632661 1866 consensus_queue.cc:237] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.651856 1878 raft_consensus.cc:493] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.652004 1878 raft_consensus.cc:515] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.652673 1878 leader_election.cc:290] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.654093 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5683d7366de84653ace56f0edcd3818c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.654970 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5683d7366de84653ace56f0edcd3818c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:04.606853 1684 tablet_loader.cc:96] loaded metadata for tablet 37765950e67d466683551f67a35e9bdb (table default.Table_119772d89d5f4afa8108d78252d81fe6 [id=129e84f069a74fe9b8ee67eaeba930a3])
I20260505 07:54:04.655443 988 leader_election.cc:304] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:04.655932 1878 raft_consensus.cc:2749] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.660470 1684 tablet_loader.cc:96] loaded metadata for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5 [id=c10b484439c443368141ceb4cf43365f])
I20260505 07:54:04.661969 1684 tablet_loader.cc:96] loaded metadata for tablet 3f869abaa31e414ea0625e475db08e64 (table default.Table_c1551c1a06c34c5dae05b3d5f1109d43 [id=276ccf1ef8a9474dbb936d880f10805e])
I20260505 07:54:04.662390 1684 tablet_loader.cc:96] loaded metadata for tablet 411025888b55405586b128d5bc0f2021 (table default.Table_a4147e267b5f45e78812bde816ae7c8d [id=0b6c6c33d8eb41dd9b351e0d4f0e8bcd])
I20260505 07:54:04.664927 1684 tablet_loader.cc:96] loaded metadata for tablet 42360f8ee4ff47c2af8d287f792207de (table default.Table_829cc346ac834637ab2e2198944a8550 [id=2c91868ddba84103b9a5c78201842a9e])
I20260505 07:54:04.665848 1684 tablet_loader.cc:96] loaded metadata for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52])
I20260505 07:54:04.666345 1176 tablet_bootstrap.cc:492] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.666536 1176 ts_tablet_manager.cc:1403] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.052s user 0.015s sys 0.011s
I20260505 07:54:04.667411 1176 raft_consensus.cc:359] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.667658 1176 raft_consensus.cc:385] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.667761 1176 raft_consensus.cc:740] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.667994 1176 consensus_queue.cc:260] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.669708 1176 ts_tablet_manager.cc:1434] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260505 07:54:04.670022 1176 tablet_bootstrap.cc:492] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.673696 1172 tablet_bootstrap.cc:492] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.673902 1172 ts_tablet_manager.cc:1403] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.066s user 0.008s sys 0.021s
I20260505 07:54:04.677752 1172 raft_consensus.cc:359] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.678012 1172 raft_consensus.cc:385] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.678121 1172 raft_consensus.cc:740] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.678429 1172 consensus_queue.cc:260] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.679220 1684 tablet_loader.cc:96] loaded metadata for tablet 429ccdb56f0b456483cf1debb5de0beb (table default.Table_a44f4d613a6e47e38e9741252ef6956a [id=1a0d30f064af4c2f8e082d5742ba6d28])
I20260505 07:54:04.679350 1172 ts_tablet_manager.cc:1434] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:54:04.679703 1172 tablet_bootstrap.cc:492] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.680737 1684 tablet_loader.cc:96] loaded metadata for tablet 46120e7a5dcb482f8f93fc2aa5219aff (table default.Table_59819a75045240de818a627f6036f8e3 [id=c019a4cae5d0458b85da8106e6819924])
I20260505 07:54:04.682034 1172 tablet_bootstrap.cc:654] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.683650 1684 tablet_loader.cc:96] loaded metadata for tablet 4662d4a3d88d4c2883a35048d6e11eb9 (table default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9 [id=3aa673bdfdb2432bb2174d4a520d7a9e])
I20260505 07:54:04.684337 1878 consensus_queue.cc:1048] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.684762 1181 tablet_bootstrap.cc:492] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:04.684964 1181 ts_tablet_manager.cc:1403] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.082s user 0.005s sys 0.001s
I20260505 07:54:04.685822 1181 raft_consensus.cc:359] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.686033 1181 raft_consensus.cc:385] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.686110 1181 raft_consensus.cc:740] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.686353 1181 consensus_queue.cc:260] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.686839 1181 ts_tablet_manager.cc:1434] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:04.687212 1181 tablet_bootstrap.cc:492] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.688856 1181 tablet_bootstrap.cc:654] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.689042 1176 tablet_bootstrap.cc:654] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.694542 1934 consensus_queue.cc:1048] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.697494 1934 raft_consensus.cc:493] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.697647 1934 raft_consensus.cc:515] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.698293 1934 leader_election.cc:290] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.698802 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0baf83b155494fddb3c25144608706f7" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.698844 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0baf83b155494fddb3c25144608706f7" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.699003 916 raft_consensus.cc:2468] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
W20260505 07:54:04.699337 717 leader_election.cc:343] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.699492 718 leader_election.cc:304] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.700009 1934 raft_consensus.cc:2804] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.700100 1934 raft_consensus.cc:493] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.700197 1934 raft_consensus.cc:3060] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.702872 1934 raft_consensus.cc:515] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.703794 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0baf83b155494fddb3c25144608706f7" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:04.703867 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0baf83b155494fddb3c25144608706f7" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:04.703995 916 raft_consensus.cc:3060] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:04.704301 717 leader_election.cc:343] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.704850 1818 fs_manager.cc:714] Time spent opening directory manager: real 0.050s user 0.008s sys 0.005s
I20260505 07:54:04.707141 916 raft_consensus.cc:2468] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.707603 718 leader_election.cc:304] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.707895 1870 raft_consensus.cc:2804] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.707996 1870 raft_consensus.cc:697] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:04.708258 1870 consensus_queue.cc:237] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.711565 1172 tablet_bootstrap.cc:492] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.711735 1172 ts_tablet_manager.cc:1403] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.001s
I20260505 07:54:04.712467 1172 raft_consensus.cc:359] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.712618 1172 raft_consensus.cc:385] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.712704 1172 raft_consensus.cc:740] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.712904 1172 consensus_queue.cc:260] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.713431 1172 ts_tablet_manager.cc:1434] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:04.713691 1172 tablet_bootstrap.cc:492] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.715390 1172 tablet_bootstrap.cc:654] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:04.715747 445 leader_election.cc:336] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565): Timed out: RequestConsensusVote RPC to 127.0.105.61:33565 timed out after 0.646s (SENT)
I20260505 07:54:04.715907 445 leader_election.cc:304] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 379cdfdecc3341ea9d7d9dc09274338d; no voters: 23c57a2c927c4c54bccecfeb3a775311, 28d3eda04f9c47ae976d19c218b6dcc5
W20260505 07:54:04.724910 991 consensus_peers.cc:597] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf76e868bbcb44b0b1887c84dd27f093. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:04.725054 1684 tablet_loader.cc:96] loaded metadata for tablet 49d7b9f5b6f74f8ea070f8ea7e545df7 (table default.Table_876e4bf0b9054b8e836299103ac03ef7 [id=a3b569958e05423588c9949eb7fe34a6])
I20260505 07:54:04.725893 1684 tablet_loader.cc:96] loaded metadata for tablet 4a8e6019406845ebb29051432a96dfd1 (table default.Table_dedf48e6bbc941f6b329f367499612f2 [id=c4d8c3d8aa554154a7015e6ff3d7b38c])
I20260505 07:54:04.726336 1684 tablet_loader.cc:96] loaded metadata for tablet 4af9def97a454f3f9b63d637db42c110 (table default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958])
I20260505 07:54:04.727026 1684 tablet_loader.cc:96] loaded metadata for tablet 4b528d6c36ac4920b6ea0a6e26cca269 (table default.Table_79ed240074b343fabbdd206872328277 [id=7cbbeae00a3d4f85bcaef70b7aaa69de])
I20260505 07:54:04.727458 1684 tablet_loader.cc:96] loaded metadata for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f])
I20260505 07:54:04.728139 1684 tablet_loader.cc:96] loaded metadata for tablet 4cb7b3fded5d4def8db8d5c1e77ddcda (table default.Table_cc284917d2aa41d9956affce2c946825 [id=231630aa148e43d1839edb43f686ad20])
I20260505 07:54:04.728621 1934 leader_election.cc:290] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.737038 2010 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260505 07:54:04.737186 1172 tablet_bootstrap.cc:492] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.737363 1172 ts_tablet_manager.cc:1403] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.001s
I20260505 07:54:04.738226 1172 raft_consensus.cc:359] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.738430 1172 raft_consensus.cc:385] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.738517 1172 raft_consensus.cc:740] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.738768 1172 consensus_queue.cc:260] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.739168 1684 tablet_loader.cc:96] loaded metadata for tablet 4e99f96a753047348371bee5c022684e (table default.Table_55f3a1ca8c53455993b486b0798f0425 [id=fb5d62bb12c5499e8953cb3b431867a9])
I20260505 07:54:04.739275 1172 ts_tablet_manager.cc:1434] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:04.739570 1684 tablet_loader.cc:96] loaded metadata for tablet 4ef22101fe014e21adea6e10f4b550e7 (table default.Table_89965306827247e99ff9f34d13e63c54 [id=567b03439f3a412aa6c1a4f4bac48f4b])
I20260505 07:54:04.740217 1684 tablet_loader.cc:96] loaded metadata for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89])
I20260505 07:54:04.740564 1684 tablet_loader.cc:96] loaded metadata for tablet 5191f9a357de4607aab6c7a8b4bcccdb (table default.Table_94e6f1ee4ab947d8bc9864ab0c8bd524 [id=f7b2051b32e240c1856be3f0802b2f3e])
I20260505 07:54:04.740845 1172 tablet_bootstrap.cc:492] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.740983 1684 tablet_loader.cc:96] loaded metadata for tablet 54e4f7f895d943639653c8c32724baf2 (table default.Table_ceaf4d54d57542928716e3056eac2bde [id=54621fbfaf6a4b4a874207bd986b77be])
I20260505 07:54:04.741333 1684 tablet_loader.cc:96] loaded metadata for tablet 55d11a9975854940a0be1693508f0841 (table default.Table_659ddda68e194bb7ae10409bbef074c6 [id=f6e2d9ac0e374944988e8ef7f88fa4be])
I20260505 07:54:04.741698 1684 tablet_loader.cc:96] loaded metadata for tablet 5683d7366de84653ace56f0edcd3818c (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3])
I20260505 07:54:04.742586 1684 tablet_loader.cc:96] loaded metadata for tablet 571257bbb7e94315b60d84d6aa4917b1 (table default.Table_4ab0ac01b8bf49ef96683dca41dfadd4 [id=6cb8bc8043474bad9f4278b7eef7c71a])
I20260505 07:54:04.742446 2012 log_block_manager.cc:3869] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260505 07:54:04.742971 1684 tablet_loader.cc:96] loaded metadata for tablet 59a3cd77808c4e858ea2ef9ed2a28c38 (table default.Table_a31277b1c1ca4863976cb9726e83f13a [id=d4aea2d5304d4113ab26a474454b05de])
I20260505 07:54:04.744467 1172 tablet_bootstrap.cc:654] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.748488 1818 fs_manager.cc:730] Time spent opening block manager: real 0.033s user 0.001s sys 0.005s
I20260505 07:54:04.748929 1818 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data,/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
uuid: "28d3eda04f9c47ae976d19c218b6dcc5"
format_stamp: "Formatted at 2026-05-05 07:53:50 on dist-test-slave-ksjb"
I20260505 07:54:04.751168 1818 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260505 07:54:04.760756 1181 tablet_bootstrap.cc:492] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:04.761999 1866 raft_consensus.cc:493] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.762223 1866 raft_consensus.cc:515] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.762418 1181 ts_tablet_manager.cc:1403] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.075s user 0.004s sys 0.015s
I20260505 07:54:04.763288 1866 leader_election.cc:290] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:04.763254 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4258df74ea0841449846cdf1f6e00b9b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.763754 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4258df74ea0841449846cdf1f6e00b9b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:04.763731 1181 raft_consensus.cc:359] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:04.763823 856 leader_election.cc:343] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.763904 1181 raft_consensus.cc:385] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.763998 1181 raft_consensus.cc:740] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.764215 1181 consensus_queue.cc:260] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:04.764446 854 leader_election.cc:343] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.764570 854 leader_election.cc:304] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:04.764777 1181 ts_tablet_manager.cc:1434] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:04.765678 1866 raft_consensus.cc:2749] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.771989 1181 tablet_bootstrap.cc:492] T 46120e7a5dcb482f8f93fc2aa5219aff P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.774850 1172 tablet_bootstrap.cc:492] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.775529 1172 ts_tablet_manager.cc:1403] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.036s user 0.004s sys 0.004s
I20260505 07:54:04.776921 1932 consensus_queue.cc:1048] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260505 07:54:04.779188 1172 raft_consensus.cc:359] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.780041 1172 raft_consensus.cc:385] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.780436 1172 raft_consensus.cc:740] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.781328 1172 consensus_queue.cc:260] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.783893 1172 ts_tablet_manager.cc:1434] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.008s user 0.001s sys 0.002s
I20260505 07:54:04.784641 1181 tablet_bootstrap.cc:654] T 46120e7a5dcb482f8f93fc2aa5219aff P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.785159 1172 tablet_bootstrap.cc:492] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.787182 1878 raft_consensus.cc:493] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.787372 1878 raft_consensus.cc:515] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.788008 1878 leader_election.cc:290] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.788740 1684 tablet_loader.cc:96] loaded metadata for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_268b9129e7fb4560bcbbb7e5427975fa [id=2c06ff8eed31432e87f08e9b59049652])
I20260505 07:54:04.777158 1870 raft_consensus.cc:493] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.794198 1181 tablet_bootstrap.cc:492] T 46120e7a5dcb482f8f93fc2aa5219aff P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:04.794566 1181 ts_tablet_manager.cc:1403] T 46120e7a5dcb482f8f93fc2aa5219aff P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.006s
I20260505 07:54:04.798350 1181 raft_consensus.cc:359] T 46120e7a5dcb482f8f93fc2aa5219aff P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.798983 1181 raft_consensus.cc:385] T 46120e7a5dcb482f8f93fc2aa5219aff P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.799268 1181 raft_consensus.cc:740] T 46120e7a5dcb482f8f93fc2aa5219aff P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.799784 1181 consensus_queue.cc:260] T 46120e7a5dcb482f8f93fc2aa5219aff P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.800120 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e62b7af9b847b7a9005e4345905c0d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.800377 920 raft_consensus.cc:2393] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:04.801805 1181 ts_tablet_manager.cc:1434] T 46120e7a5dcb482f8f93fc2aa5219aff P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260505 07:54:04.803395 1181 tablet_bootstrap.cc:492] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.806437 1181 tablet_bootstrap.cc:654] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.809386 1866 consensus_queue.cc:1048] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.812914 1181 tablet_bootstrap.cc:492] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:04.813789 1181 ts_tablet_manager.cc:1403] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.003s
I20260505 07:54:04.815028 1181 raft_consensus.cc:359] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.816459 1181 raft_consensus.cc:385] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.816820 1181 raft_consensus.cc:740] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.817296 1181 consensus_queue.cc:260] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.819794 1172 tablet_bootstrap.cc:654] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.821574 1181 ts_tablet_manager.cc:1434] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:54:04.822335 1172 tablet_bootstrap.cc:492] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.822525 1172 ts_tablet_manager.cc:1403] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.038s user 0.001s sys 0.004s
I20260505 07:54:04.822870 1181 tablet_bootstrap.cc:492] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.824800 1684 tablet_loader.cc:96] loaded metadata for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6])
I20260505 07:54:04.825292 1684 tablet_loader.cc:96] loaded metadata for tablet 66be7b6c33a342d6890e459d90d40583 (table default.Table_28c935edd66e47959a2df29d39dfe13c [id=fe9a0701ecd44cd28915885958b3b3c6])
I20260505 07:54:04.825320 1181 tablet_bootstrap.cc:654] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.825582 1172 raft_consensus.cc:359] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.825809 1172 raft_consensus.cc:385] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.825730 1684 tablet_loader.cc:96] loaded metadata for tablet 66f069982138460da3b7ae7a0b4e3833 (table default.Table_5e1ab4e85d4b46d8a5d98434da77c9a8 [id=e86701e2ef8c4075877c58927318622f])
I20260505 07:54:04.825899 1172 raft_consensus.cc:740] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.826289 1684 tablet_loader.cc:96] loaded metadata for tablet 676e2109e59749b58931d597ce1beb9c (table default.Table_fcec571b3c7140669fe8d8c983cd616f [id=f47edf0bb7a14132a37d1f820082f0b9])
I20260505 07:54:04.826180 1172 consensus_queue.cc:260] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.830461 1181 tablet_bootstrap.cc:492] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:04.830768 1181 ts_tablet_manager.cc:1403] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260505 07:54:04.831925 1181 raft_consensus.cc:359] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.832295 1181 raft_consensus.cc:385] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.832638 1181 raft_consensus.cc:740] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.832948 1181 consensus_queue.cc:260] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.833441 1181 ts_tablet_manager.cc:1434] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260505 07:54:04.833770 1181 tablet_bootstrap.cc:492] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.835307 1181 tablet_bootstrap.cc:654] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.838131 1779 consensus_queue.cc:1048] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.838963 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "83e62b7af9b847b7a9005e4345905c0d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:04.840018 1878 raft_consensus.cc:493] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.840167 1878 raft_consensus.cc:515] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.840818 1049 raft_consensus.cc:3060] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.841416 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f2f46d0e7840458cb992ce2a2dbea2" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.841600 913 raft_consensus.cc:2468] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:04.842254 991 leader_election.cc:304] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:04.842548 1580 raft_consensus.cc:2804] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.842638 1580 raft_consensus.cc:493] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.842732 1580 raft_consensus.cc:3060] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.840826 1878 leader_election.cc:290] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.845021 1181 tablet_bootstrap.cc:492] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:04.845191 1181 ts_tablet_manager.cc:1403] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.004s
I20260505 07:54:04.846403 1580 raft_consensus.cc:515] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.846988 988 leader_election.cc:304] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:04.847291 1878 raft_consensus.cc:3060] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.847886 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f2f46d0e7840458cb992ce2a2dbea2" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:04.848109 913 raft_consensus.cc:3060] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.849875 1878 raft_consensus.cc:2749] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.850927 1931 consensus_queue.cc:1048] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:04.851732 1580 leader_election.cc:290] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.851737 1866 raft_consensus.cc:493] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.851897 1866 raft_consensus.cc:515] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.852142 913 raft_consensus.cc:2468] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:04.852659 1866 leader_election.cc:290] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:04.852638 991 leader_election.cc:304] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:04.852957 1580 raft_consensus.cc:2804] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.853125 1580 raft_consensus.cc:697] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:04.853484 1580 consensus_queue.cc:237] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.854614 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dc6df9da5401384a6fc5de868b41c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:04.856037 854 leader_election.cc:343] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.856868 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dc6df9da5401384a6fc5de868b41c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.857510 1181 raft_consensus.cc:359] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.857697 1181 raft_consensus.cc:385] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.857805 1181 raft_consensus.cc:740] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.858027 1181 consensus_queue.cc:260] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.858286 1779 raft_consensus.cc:493] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:04.858387 856 leader_election.cc:343] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.858487 856 leader_election.cc:304] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:04.859021 1176 tablet_bootstrap.cc:492] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.859195 1176 ts_tablet_manager.cc:1403] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.189s user 0.008s sys 0.041s
I20260505 07:54:04.859169 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f2f46d0e7840458cb992ce2a2dbea2" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:04.859689 1866 raft_consensus.cc:2749] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.860697 1181 ts_tablet_manager.cc:1434] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.015s user 0.000s sys 0.003s
I20260505 07:54:04.860986 1181 tablet_bootstrap.cc:492] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:04.861615 1779 raft_consensus.cc:515] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.862769 1176 raft_consensus.cc:359] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.862939 1176 raft_consensus.cc:385] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.863786 1779 leader_election.cc:290] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.863761 1176 raft_consensus.cc:740] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.864063 1176 consensus_queue.cc:260] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.863202 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f2f46d0e7840458cb992ce2a2dbea2" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:04.864671 1176 ts_tablet_manager.cc:1434] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:54:04.864952 1176 tablet_bootstrap.cc:492] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.865454 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0459f2d45a2d44f1a73e0a0fff53866c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.865670 913 raft_consensus.cc:2468] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:04.866178 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0459f2d45a2d44f1a73e0a0fff53866c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.866391 718 leader_election.cc:304] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:04.866395 1055 raft_consensus.cc:2468] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:04.866794 1176 tablet_bootstrap.cc:654] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.867388 1779 raft_consensus.cc:2804] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.867518 1779 raft_consensus.cc:493] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.867668 1779 raft_consensus.cc:3060] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.872095 1779 raft_consensus.cc:515] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.874671 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0459f2d45a2d44f1a73e0a0fff53866c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:04.874732 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0459f2d45a2d44f1a73e0a0fff53866c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:04.875324 1055 raft_consensus.cc:3060] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.875331 913 raft_consensus.cc:3060] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.875335 1181 tablet_bootstrap.cc:654] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.878142 1055 raft_consensus.cc:2468] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:04.878711 717 leader_election.cc:304] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:04.879010 1864 raft_consensus.cc:2804] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.879185 1864 raft_consensus.cc:697] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:04.879470 1864 consensus_queue.cc:237] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.880051 1779 leader_election.cc:290] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.886516 1176 tablet_bootstrap.cc:492] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.887501 1176 ts_tablet_manager.cc:1403] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.000s
I20260505 07:54:04.891098 1870 raft_consensus.cc:515] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.893123 1176 raft_consensus.cc:359] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:04.894311 856 consensus_peers.cc:597] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:04.894338 1176 raft_consensus.cc:385] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.894641 1176 raft_consensus.cc:740] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.895486 1176 consensus_queue.cc:260] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.899026 1176 ts_tablet_manager.cc:1434] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20260505 07:54:04.901197 1176 tablet_bootstrap.cc:492] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.903403 781 raft_consensus.cc:3060] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.906680 781 raft_consensus.cc:2468] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:04.908856 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f2f46d0e7840458cb992ce2a2dbea2" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:04.911720 1176 tablet_bootstrap.cc:654] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.923484 1172 ts_tablet_manager.cc:1434] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.101s user 0.001s sys 0.002s
I20260505 07:54:04.927321 913 raft_consensus.cc:2468] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:04.933360 856 consensus_peers.cc:597] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:04.933413 1176 tablet_bootstrap.cc:492] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.933562 1176 ts_tablet_manager.cc:1403] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.001s
I20260505 07:54:04.934321 1176 raft_consensus.cc:359] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.934535 1176 raft_consensus.cc:385] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.934630 1176 raft_consensus.cc:740] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.934983 1176 consensus_queue.cc:260] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.935253 1172 tablet_bootstrap.cc:492] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.935638 1176 ts_tablet_manager.cc:1434] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:04.936398 1870 leader_election.cc:290] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:04.936759 1176 tablet_bootstrap.cc:492] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.937217 1172 tablet_bootstrap.cc:654] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.938601 1176 tablet_bootstrap.cc:654] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:04.940923 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0f2f46d0e7840458cb992ce2a2dbea2" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:04.941215 913 raft_consensus.cc:2393] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:04.941793 718 leader_election.cc:304] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.942435 1864 raft_consensus.cc:2749] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.942939 1684 tablet_loader.cc:96] loaded metadata for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_38d26c09769b42159b828ec5bb429075 [id=fc9aba6852ee42cc83697f86e0e69c5e])
I20260505 07:54:04.950663 1906 raft_consensus.cc:493] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.950811 1906 raft_consensus.cc:515] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.951894 1906 leader_election.cc:290] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:04.952819 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5191f9a357de4607aab6c7a8b4bcccdb" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:04.953300 854 leader_election.cc:343] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.953683 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5191f9a357de4607aab6c7a8b4bcccdb" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:04.954200 856 leader_election.cc:343] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.954313 856 leader_election.cc:304] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:04.954525 1864 raft_consensus.cc:493] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.954733 1906 raft_consensus.cc:2749] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.954720 1864 raft_consensus.cc:515] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.955407 1864 leader_election.cc:290] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:04.956223 718 leader_election.cc:343] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 3d83e9d4243e409794ffe8a342bce6b3
W20260505 07:54:04.956847 717 leader_election.cc:343] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 3d83e9d4243e409794ffe8a342bce6b3
I20260505 07:54:04.956990 717 leader_election.cc:304] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.957280 1864 raft_consensus.cc:2749] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:04.959983 1684 tablet_loader.cc:96] loaded metadata for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_5f0d98b4110b4596a6de3749d10302c9 [id=5bf31c7aad3d40b0a73a40a14ca8c04d])
I20260505 07:54:04.960310 1906 raft_consensus.cc:493] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.960453 1906 raft_consensus.cc:515] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.961074 1906 leader_election.cc:290] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:04.961233 1684 tablet_loader.cc:96] loaded metadata for tablet 6ea1d3829ede4b5582a022b64265df7e (table default.Table_4ce294c5bd3f49699d1e194c53ff9c07 [id=0ca5072e233148099ba64cca861c144c])
I20260505 07:54:04.961627 1684 tablet_loader.cc:96] loaded metadata for tablet 6efb07cb60f64dfead739a3ca1343df9 (table default.Table_ed41728f809d4b0eb56670c1410d14ac [id=6e6683c2d09e4532aa3a56246c56bfd4])
I20260505 07:54:04.962054 1684 tablet_loader.cc:96] loaded metadata for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491])
I20260505 07:54:04.962042 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a61c1d03d544c8aa01537ead97f83c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:04.962242 776 raft_consensus.cc:2468] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:04.962456 1684 tablet_loader.cc:96] loaded metadata for tablet 6fe19306f7df49fdb771d21f6560e697 (table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93])
I20260505 07:54:04.962823 854 leader_election.cc:304] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:04.963147 1684 tablet_loader.cc:96] loaded metadata for tablet 70051011024f4971bd0166d335d32393 (table default.Table_6b9d99bdf8624bba90ba61af22f36dc2 [id=6fa686898f454d2aa691814ad1284d0f])
I20260505 07:54:04.963157 1906 raft_consensus.cc:2804] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.963260 1906 raft_consensus.cc:493] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.963376 1906 raft_consensus.cc:3060] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.963814 1684 tablet_loader.cc:96] loaded metadata for tablet 70404184502a4394a737518220f2124f (table default.Table_cf3b3e8080ba449a9f92b5175e2193b1 [id=22feb370028a4bd59568731eaa7b090c])
I20260505 07:54:04.964177 1684 tablet_loader.cc:96] loaded metadata for tablet 7071df211bd34b86a751142bae3271b2 (table default.Table_93d07f8983d94584a9c9700054d460e6 [id=1c2e08372ceb44e7945a4abb176da05e])
I20260505 07:54:04.964804 1684 tablet_loader.cc:96] loaded metadata for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4])
I20260505 07:54:04.965408 1684 tablet_loader.cc:96] loaded metadata for tablet 72da22d790624b07bfca6f6b6301c925 (table default.Table_201f8409684e49be8bc5cd5249606250 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1])
I20260505 07:54:04.965736 1684 tablet_loader.cc:96] loaded metadata for tablet 73042a7ff5fb43f7a781b4f98f4f4020 (table default.Table_40605e9a72634999bc80f464976c5dac [id=c1d7934526274ef09104dbff5eaacb1e])
I20260505 07:54:04.966445 1906 raft_consensus.cc:515] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.966514 1684 tablet_loader.cc:96] loaded metadata for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_95fae21889c84b61ad67610d209031b2 [id=b131fb4265684cd2b6506a7abeb29871])
I20260505 07:54:04.967454 1684 tablet_loader.cc:96] loaded metadata for tablet 7b8851ec03694289ae33a86f2faa4b09 (table default.Table_f99f927cdc404f66849fad37ee1f1445 [id=3637ceb2e2d8469ebc7c9f871c73c806])
I20260505 07:54:04.967813 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a61c1d03d544c8aa01537ead97f83c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:04.968040 1055 raft_consensus.cc:3060] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.971381 1055 raft_consensus.cc:2468] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:04.971647 1848 raft_consensus.cc:493] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:04.971793 1848 raft_consensus.cc:515] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.971987 856 leader_election.cc:304] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:04.972335 1909 raft_consensus.cc:2804] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.972493 1909 raft_consensus.cc:697] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:04.972782 1909 consensus_queue.cc:237] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.973275 1172 tablet_bootstrap.cc:492] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:04.973830 1172 ts_tablet_manager.cc:1403] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.039s user 0.008s sys 0.002s
I20260505 07:54:04.972491 1848 leader_election.cc:290] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:04.974993 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ae6404dd64bd8bf4898768e94e4f1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:04.975455 856 leader_election.cc:343] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.976614 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ae6404dd64bd8bf4898768e94e4f1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:04.976392 1172 raft_consensus.cc:359] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.976799 781 raft_consensus.cc:2468] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:04.977238 854 leader_election.cc:304] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.977262 1172 raft_consensus.cc:385] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.977698 1848 raft_consensus.cc:2804] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:04.977865 1848 raft_consensus.cc:493] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:04.977952 1848 raft_consensus.cc:3060] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.977934 1172 raft_consensus.cc:740] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.979497 1172 consensus_queue.cc:260] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.981215 781 raft_consensus.cc:3060] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.980861 1848 raft_consensus.cc:515] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.983392 1172 ts_tablet_manager.cc:1434] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.009s user 0.003s sys 0.001s
I20260505 07:54:04.985013 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ae6404dd64bd8bf4898768e94e4f1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:04.985502 856 leader_election.cc:343] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:04.986155 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ae6404dd64bd8bf4898768e94e4f1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:04.986315 776 raft_consensus.cc:3060] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.987037 1848 leader_election.cc:290] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:04.987082 1172 tablet_bootstrap.cc:492] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:04.990101 776 raft_consensus.cc:2468] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:04.990469 854 leader_election.cc:304] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:04.990688 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a61c1d03d544c8aa01537ead97f83c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:04.990834 778 raft_consensus.cc:3060] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:04.991005 1848 raft_consensus.cc:2804] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:04.991401 1848 raft_consensus.cc:697] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:04.991761 1848 consensus_queue.cc:237] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:04.992940 1176 tablet_bootstrap.cc:492] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:04.993111 1176 ts_tablet_manager.cc:1403] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.056s user 0.009s sys 0.018s
I20260505 07:54:04.993660 778 raft_consensus.cc:2468] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:04.993842 1906 leader_election.cc:290] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:04.994021 1176 raft_consensus.cc:359] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.994230 1176 raft_consensus.cc:385] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:04.994331 1176 raft_consensus.cc:740] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:04.994566 1176 consensus_queue.cc:260] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:04.995093 1176 ts_tablet_manager.cc:1434] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:04.995393 1176 tablet_bootstrap.cc:492] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:04.998045 1176 tablet_bootstrap.cc:654] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.003939 1172 tablet_bootstrap.cc:654] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.004292 1864 raft_consensus.cc:493] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.004485 1864 raft_consensus.cc:515] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.004877 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1a61c1d03d544c8aa01537ead97f83c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.005193 1864 leader_election.cc:290] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:05.005549 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42360f8ee4ff47c2af8d287f792207de" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.005795 913 raft_consensus.cc:2468] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:05.006222 718 leader_election.cc:304] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:05.006518 1864 raft_consensus.cc:2804] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.006654 1864 raft_consensus.cc:493] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.006772 1864 raft_consensus.cc:3060] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.009552 1053 raft_consensus.cc:2376] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1. Re-sending same reply.
I20260505 07:54:05.006807 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42360f8ee4ff47c2af8d287f792207de" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.008880 1804 raft_consensus.cc:493] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.011379 1804 raft_consensus.cc:515] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.011991 1804 leader_election.cc:290] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.012377 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02412333633a4cdebe95a144fa6c363c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.012753 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02412333633a4cdebe95a144fa6c363c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.012704 1864 raft_consensus.cc:515] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:05.012825 717 leader_election.cc:343] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.012979 913 raft_consensus.cc:2468] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:05.013350 1176 tablet_bootstrap.cc:492] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:05.013450 991 leader_election.cc:304] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:05.013504 1176 ts_tablet_manager.cc:1403] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.003s
I20260505 07:54:05.014186 1804 raft_consensus.cc:2804] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.014272 1804 raft_consensus.cc:493] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.014251 1176 raft_consensus.cc:359] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.014385 1804 raft_consensus.cc:3060] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.014430 1176 raft_consensus.cc:385] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.014520 1176 raft_consensus.cc:740] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.014760 778 raft_consensus.cc:2468] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:05.014787 1864 leader_election.cc:290] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:05.014945 1176 consensus_queue.cc:260] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.015697 1176 ts_tablet_manager.cc:1434] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:05.015985 1176 tablet_bootstrap.cc:492] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:05.018362 1176 tablet_bootstrap.cc:654] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.018307 1804 raft_consensus.cc:515] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.019533 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02412333633a4cdebe95a144fa6c363c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:05.019697 920 raft_consensus.cc:3060] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.022408 920 raft_consensus.cc:2468] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:05.022780 991 leader_election.cc:304] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:05.023062 1878 raft_consensus.cc:2804] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:05.023192 1878 raft_consensus.cc:697] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:05.023456 1878 consensus_queue.cc:237] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.025748 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42360f8ee4ff47c2af8d287f792207de" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:05.025962 920 raft_consensus.cc:3060] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.028085 1804 leader_election.cc:290] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.028923 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42360f8ee4ff47c2af8d287f792207de" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:05.029647 1909 raft_consensus.cc:493] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:05.030055 717 leader_election.cc:343] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.030174 1909 raft_consensus.cc:515] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.032644 1909 leader_election.cc:290] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:05.033059 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bf3a6bc0ac42a99a4181f820e3ca87" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.033174 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bf3a6bc0ac42a99a4181f820e3ca87" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:05.033494 856 leader_election.cc:343] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:05.033814 854 leader_election.cc:343] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.033965 854 leader_election.cc:304] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.029937 1906 raft_consensus.cc:493] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.034473 1906 raft_consensus.cc:515] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.035445 1906 leader_election.cc:290] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:05.035674 1906 raft_consensus.cc:2749] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.036114 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1001dd888ecd4166ba217506739f71e8" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.036253 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1001dd888ecd4166ba217506739f71e8" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:05.036608 856 leader_election.cc:343] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:05.036898 854 leader_election.cc:343] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.037039 854 leader_election.cc:304] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.039690 1909 raft_consensus.cc:2749] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.040017 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02412333633a4cdebe95a144fa6c363c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:05.040279 781 raft_consensus.cc:3060] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.043040 781 raft_consensus.cc:2468] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:05.043881 920 raft_consensus.cc:2468] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:05.044402 718 leader_election.cc:304] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:05.050081 1864 raft_consensus.cc:2804] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:05.050244 1864 raft_consensus.cc:697] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:05.050508 1864 consensus_queue.cc:237] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.058995 1684 tablet_loader.cc:96] loaded metadata for tablet 839b963b04b24cf8af8d358a69af4f39 (table default.Table_abc41dd87e7a45c1b52bf036a9dfab27 [id=eb2e83fd347c4d2583a6fdf9bac20d53])
I20260505 07:54:05.063292 1181 tablet_bootstrap.cc:492] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.064021 1181 ts_tablet_manager.cc:1403] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.203s user 0.049s sys 0.056s
I20260505 07:54:05.065171 1181 raft_consensus.cc:359] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.066890 1176 tablet_bootstrap.cc:492] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:05.067111 1176 ts_tablet_manager.cc:1403] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.009s
I20260505 07:54:05.068166 1176 raft_consensus.cc:359] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.068501 1181 raft_consensus.cc:385] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.068666 1176 raft_consensus.cc:385] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.068817 1176 raft_consensus.cc:740] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.068948 1181 raft_consensus.cc:740] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.069265 1176 consensus_queue.cc:260] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.069708 1181 consensus_queue.cc:260] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.070487 1176 ts_tablet_manager.cc:1434] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:05.071686 1181 ts_tablet_manager.cc:1434] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:54:05.072816 1181 tablet_bootstrap.cc:492] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.073462 1172 tablet_bootstrap.cc:492] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
W20260505 07:54:05.063040 1317 consensus_peers.cc:597] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 -> Peer 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401): Couldn't send request to peer 28d3eda04f9c47ae976d19c218b6dcc5. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:05.074404 1172 ts_tablet_manager.cc:1403] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.088s user 0.001s sys 0.012s
I20260505 07:54:05.074512 1684 tablet_loader.cc:96] loaded metadata for tablet 83e62b7af9b847b7a9005e4345905c0d (table default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d])
I20260505 07:54:05.075261 1181 tablet_bootstrap.cc:654] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.075465 1684 tablet_loader.cc:96] loaded metadata for tablet 840eebf9e7404bf7807b7e51996957e5 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3])
I20260505 07:54:05.077639 1176 tablet_bootstrap.cc:492] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:05.076823 1172 raft_consensus.cc:359] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.079521 1176 tablet_bootstrap.cc:654] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.079510 1172 raft_consensus.cc:385] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.081054 1172 raft_consensus.cc:740] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.085362 1818 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20260505 07:54:05.084730 1172 consensus_queue.cc:260] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.088479 1684 tablet_loader.cc:96] loaded metadata for tablet 84faf2e3d91e4cc394a3411c5b6c0bc7 (table default.Table_f8b3c6c6274f468fb9f56dad70e9043c [id=c91f3cdfd7534217ba6f4c985afd18e4])
I20260505 07:54:05.089740 1172 ts_tablet_manager.cc:1434] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260505 07:54:05.091439 1176 tablet_bootstrap.cc:492] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:05.091598 1176 ts_tablet_manager.cc:1403] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.000s
I20260505 07:54:05.092338 1172 tablet_bootstrap.cc:492] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.092414 1176 raft_consensus.cc:359] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.092605 1176 raft_consensus.cc:385] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.092695 1176 raft_consensus.cc:740] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.092921 1176 consensus_queue.cc:260] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.093474 1818 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260505 07:54:05.093640 1176 ts_tablet_manager.cc:1434] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:05.093832 1818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260505 07:54:05.093952 1176 tablet_bootstrap.cc:492] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:05.096266 1181 tablet_bootstrap.cc:492] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.096467 1181 ts_tablet_manager.cc:1403] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.009s
I20260505 07:54:05.097275 1181 raft_consensus.cc:359] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.097506 1181 raft_consensus.cc:385] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.097635 1181 raft_consensus.cc:740] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.097956 1181 consensus_queue.cc:260] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.099232 1181 ts_tablet_manager.cc:1434] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:05.100910 1172 tablet_bootstrap.cc:654] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.108803 1684 tablet_loader.cc:96] loaded metadata for tablet 85d485ae2d1d4d76a6a5d988cec8291a (table default.Table_394242d5ee184ae2b22cc355a2160fb9 [id=6e71d3f928fc45c191f64a945980a107])
I20260505 07:54:05.109367 1181 tablet_bootstrap.cc:492] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.113034 1176 tablet_bootstrap.cc:654] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.114912 1684 tablet_loader.cc:96] loaded metadata for tablet 898ebd1a70984cc1b9a71009b1070bfe (table default.Table_a11d31ae6a634b5693db8de72ca33cf8 [id=256562aafb76429fb120899d8af6704d])
I20260505 07:54:05.116592 1864 raft_consensus.cc:493] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.116747 1864 raft_consensus.cc:515] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.117093 1172 tablet_bootstrap.cc:492] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.117237 1172 ts_tablet_manager.cc:1403] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.000s
I20260505 07:54:05.117409 1864 leader_election.cc:290] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.117902 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8c9eaaef250461ca7a37d0cf42b07ee" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.117908 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8c9eaaef250461ca7a37d0cf42b07ee" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.118119 1172 raft_consensus.cc:359] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.118336 1172 raft_consensus.cc:385] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.118432 1172 raft_consensus.cc:740] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:54:05.118439 717 leader_election.cc:343] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.118120 920 raft_consensus.cc:2468] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:05.118690 1172 consensus_queue.cc:260] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.118932 1181 tablet_bootstrap.cc:654] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.119285 1172 ts_tablet_manager.cc:1434] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:05.119441 718 leader_election.cc:304] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:05.119571 1172 tablet_bootstrap.cc:492] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.119745 1864 raft_consensus.cc:2804] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.119850 1864 raft_consensus.cc:493] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.119930 1864 raft_consensus.cc:3060] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.122134 1172 tablet_bootstrap.cc:654] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.123329 1864 raft_consensus.cc:515] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.125212 1864 leader_election.cc:290] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.126963 1684 tablet_loader.cc:96] loaded metadata for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b])
I20260505 07:54:05.127151 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8c9eaaef250461ca7a37d0cf42b07ee" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:05.127775 717 leader_election.cc:343] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.128300 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8c9eaaef250461ca7a37d0cf42b07ee" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:05.128582 920 raft_consensus.cc:3060] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.130545 1864 consensus_queue.cc:1048] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.131922 920 raft_consensus.cc:2468] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:05.135293 718 leader_election.cc:304] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:05.135720 1864 raft_consensus.cc:2804] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:05.136184 1864 raft_consensus.cc:697] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:05.136366 1878 raft_consensus.cc:493] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.136533 1878 raft_consensus.cc:515] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.137181 1878 leader_election.cc:290] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.137835 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d4bc3154704ba2bfde83bca10f812d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.138908 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62d4bc3154704ba2bfde83bca10f812d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.139142 781 raft_consensus.cc:2393] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:05.139874 988 leader_election.cc:304] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.140403 1878 raft_consensus.cc:3060] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.145406 1878 raft_consensus.cc:2749] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.146430 1848 raft_consensus.cc:493] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.146613 1848 raft_consensus.cc:515] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.147241 1848 leader_election.cc:290] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:05.148089 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7001482d90364dbdb8b9c5bfb98e1cd4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.148088 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7001482d90364dbdb8b9c5bfb98e1cd4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:05.148766 854 leader_election.cc:343] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:05.149176 856 leader_election.cc:343] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.149308 856 leader_election.cc:304] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.149765 1848 raft_consensus.cc:2749] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.149178 1864 consensus_queue.cc:237] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.154719 1684 tablet_loader.cc:96] loaded metadata for tablet 8eb7eaff8837485a8ae3ccb44c60471d (table default.Table_66f28ba637f24448816a64c7d316edee [id=266ff796c7a74c23816dd9c422e9d005])
I20260505 07:54:05.156005 1684 tablet_loader.cc:96] loaded metadata for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79])
I20260505 07:54:05.156731 1684 tablet_loader.cc:96] loaded metadata for tablet 934cea6666b1420eb12fe4510d2466fe (table default.Table_f3fff69ccbc54a58bdad7a8b5489d525 [id=c0d6550f259540e2943e2378fc4bfde5])
I20260505 07:54:05.168802 1176 tablet_bootstrap.cc:492] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:05.169013 1176 ts_tablet_manager.cc:1403] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.075s user 0.003s sys 0.004s
I20260505 07:54:05.170075 1055 raft_consensus.cc:3060] T 46120e7a5dcb482f8f93fc2aa5219aff P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.174659 1779 consensus_queue.cc:1048] T 46120e7a5dcb482f8f93fc2aa5219aff P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.177106 1848 raft_consensus.cc:493] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.177286 1848 raft_consensus.cc:515] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.177883 1848 leader_election.cc:290] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:05.178591 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d11bd19f074bbab101448b45426b7a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.178956 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d11bd19f074bbab101448b45426b7a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:05.179630 854 leader_election.cc:343] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:05.179997 856 leader_election.cc:343] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.180135 856 leader_election.cc:304] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.181324 1906 raft_consensus.cc:2749] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.182675 1906 consensus_queue.cc:1048] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.183465 1684 tablet_loader.cc:96] loaded metadata for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6])
I20260505 07:54:05.184546 1684 tablet_loader.cc:96] loaded metadata for tablet 9ac029d9a3c540f58f393e79bf7be30e (table default.Table_f40f1996d8454947afd1ff280e2b5fc1 [id=541feb1163924c0ca3fb40a84ced3826])
I20260505 07:54:05.187618 1932 consensus_queue.cc:1048] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.187918 1181 tablet_bootstrap.cc:492] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.188436 1181 ts_tablet_manager.cc:1403] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.080s user 0.003s sys 0.009s
I20260505 07:54:05.189782 1684 tablet_loader.cc:96] loaded metadata for tablet 9b5840ac386e47d6a1229015549f9d92 (table default.Table_194f3e5aa086410eb3bd64b872d2f8af [id=1a7d610787c44274a6036482200780cf])
I20260505 07:54:05.193715 1931 consensus_queue.cc:1048] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.191797 1181 raft_consensus.cc:359] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.194670 1181 raft_consensus.cc:385] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.198740 1181 raft_consensus.cc:740] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.199985 1684 tablet_loader.cc:96] loaded metadata for tablet 9c4fc18f6faa488882d9bffb30793cdd (table default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f])
I20260505 07:54:05.200138 1848 raft_consensus.cc:493] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.200276 1848 raft_consensus.cc:515] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.200562 1684 tablet_loader.cc:96] loaded metadata for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb])
I20260505 07:54:05.200910 1848 leader_election.cc:290] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:05.200371 1181 consensus_queue.cc:260] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.201771 1684 tablet_loader.cc:96] loaded metadata for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_7e65acc795454f48ad6b8d0dc5dc163d [id=bccdbf7ecfdd40c59dc52c4562f88728])
I20260505 07:54:05.202066 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e4c92b7c3140fdb2d6a290aaf0ad55" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.202301 1181 ts_tablet_manager.cc:1434] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.014s user 0.000s sys 0.004s
W20260505 07:54:05.202762 856 leader_election.cc:343] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.203490 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e4c92b7c3140fdb2d6a290aaf0ad55" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.203776 781 raft_consensus.cc:2468] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:05.204761 854 leader_election.cc:304] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:05.205425 1931 consensus_queue.cc:1048] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.207070 1181 tablet_bootstrap.cc:492] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.209316 1848 raft_consensus.cc:2804] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.209437 1848 raft_consensus.cc:493] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.209537 1848 raft_consensus.cc:3060] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.211261 1684 tablet_loader.cc:96] loaded metadata for tablet a890d5e3f1af49b5aaa28c6c51cb3874 (table default.Table_eb1e569ff0a74997b956acbad82f6c49 [id=e996e55aed7c4c08bb2fd8a6fc358155])
I20260505 07:54:05.211905 1684 tablet_loader.cc:96] loaded metadata for tablet a9b7c6e9463342a5a73af12da4447f89 (table default.Table_0393e87d0a314ee898b11935e2f6666d [id=ab93bfbabbad4fad893f7a6dd595540f])
I20260505 07:54:05.217890 1870 consensus_queue.cc:1048] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.224987 1181 tablet_bootstrap.cc:654] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.231305 1848 raft_consensus.cc:515] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.237180 1848 leader_election.cc:290] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:05.237336 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e4c92b7c3140fdb2d6a290aaf0ad55" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:05.237524 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e4c92b7c3140fdb2d6a290aaf0ad55" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:05.237560 781 raft_consensus.cc:3060] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:05.238029 856 leader_election.cc:343] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.241408 781 raft_consensus.cc:2468] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:05.242189 854 leader_election.cc:304] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:05.242690 1906 raft_consensus.cc:493] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.242816 1906 raft_consensus.cc:515] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.243417 1906 leader_election.cc:290] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:05.243477 1684 tablet_loader.cc:96] loaded metadata for tablet aa049ad4edbd466a860428c822a52724 (table default.Table_881e42cce93c46cb8c6b2ff28d713b85 [id=bc1adc573d4c43afa8c5bdd900e53d65])
I20260505 07:54:05.244124 1684 tablet_loader.cc:96] loaded metadata for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_043f3d0ae81c46a9a77b3264088fd1a9 [id=37b0844f588e413f99413fa009c1ef0c])
I20260505 07:54:05.244298 1848 raft_consensus.cc:2804] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:05.245364 1848 raft_consensus.cc:697] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:05.245637 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c6ed755fc64080a2310fe9a4e0a68b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.246734 1684 tablet_loader.cc:96] loaded metadata for tablet abaa596871bb4a82aefc317932e91d35 (table default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d [id=1fcae33a3a2d48e7b2000cc7b906c7f5])
I20260505 07:54:05.246989 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c6ed755fc64080a2310fe9a4e0a68b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:05.246621 854 leader_election.cc:343] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:05.248270 856 leader_election.cc:343] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.248415 856 leader_election.cc:304] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.248791 1906 raft_consensus.cc:2749] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.248972 1172 tablet_bootstrap.cc:492] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.249121 1684 tablet_loader.cc:96] loaded metadata for tablet accb2dedc30e494ebd22c33594cee661 (table default.Table_fa6246cc3ccd4d7082becde923f88828 [id=860ab29ce1a44ad1b2480dbdb2b40000])
I20260505 07:54:05.249344 1172 ts_tablet_manager.cc:1403] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.130s user 0.050s sys 0.046s
I20260505 07:54:05.249423 1848 consensus_queue.cc:237] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.251164 1172 raft_consensus.cc:359] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.251649 1172 raft_consensus.cc:385] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.251821 1172 raft_consensus.cc:740] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.252457 1684 tablet_loader.cc:96] loaded metadata for tablet ad5c570d4861460d8ab0c8ff8a7a3a03 (table default.Table_d17cd5686eb94c3b8993d4310f0d31f7 [id=8c0e89be595c4eb2b4d4103121cdccf0])
I20260505 07:54:05.252429 1172 consensus_queue.cc:260] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.253737 1172 ts_tablet_manager.cc:1434] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260505 07:54:05.254386 1172 tablet_bootstrap.cc:492] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.254797 1684 tablet_loader.cc:96] loaded metadata for tablet ae829daad64545e687a9528e5c38b9ed (table default.Table_06ea1b3a616b4c5aa573a1c654538879 [id=a648fb7ba63246c38e2a9f120d6d0396])
I20260505 07:54:05.256412 1181 tablet_bootstrap.cc:492] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.256701 1181 ts_tablet_manager.cc:1403] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.008s
I20260505 07:54:05.256935 1684 tablet_loader.cc:96] loaded metadata for tablet af8af6d1f5d9429083002991e3470f7b (table default.Table_a930977d21a04e229868a60d643ca173 [id=28164b2107de472ab15c913fbbc9840b])
I20260505 07:54:05.258379 1181 raft_consensus.cc:359] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.258598 1172 tablet_bootstrap.cc:654] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.258713 1181 raft_consensus.cc:385] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.258905 1181 raft_consensus.cc:740] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.259379 1181 consensus_queue.cc:260] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.260540 1684 tablet_loader.cc:96] loaded metadata for tablet b0d966bcd7f1419cbd090afc185ae78a (table default.Table_cc718440246d4ddfb49faf4602910188 [id=3ae3bc457dcb4db1b5f4360cb9e5c81c])
I20260505 07:54:05.261662 1684 tablet_loader.cc:96] loaded metadata for tablet b12d2d2b866348d781209fa787da7d7e (table default.Table_7ed6d926da574429be8db03bde4b4221 [id=e52c3ee18ce644d2b988c2e70c6307d7])
I20260505 07:54:05.261531 1181 ts_tablet_manager.cc:1434] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260505 07:54:05.262897 1181 tablet_bootstrap.cc:492] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.269716 1779 raft_consensus.cc:493] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.269886 1779 raft_consensus.cc:515] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.270149 1176 raft_consensus.cc:359] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.270332 1176 raft_consensus.cc:385] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.270510 1779 leader_election.cc:290] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:05.270545 1176 raft_consensus.cc:740] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.270839 1176 consensus_queue.cc:260] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.271528 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab3850799254dda9716fd5bf9404159" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:05.273116 717 leader_election.cc:343] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.275827 1181 tablet_bootstrap.cc:654] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.271325 1176 ts_tablet_manager.cc:1434] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.102s user 0.001s sys 0.001s
I20260505 07:54:05.280658 1176 tablet_bootstrap.cc:492] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:05.284157 1684 tablet_loader.cc:96] loaded metadata for tablet b256d072933b4566b5587224274deb66 (table default.Table_6cb216f92da74a78acd72a6132b94c1a [id=c0306ceddf024a1589f83f390b38c602])
I20260505 07:54:05.287191 1181 tablet_bootstrap.cc:492] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.287489 1181 ts_tablet_manager.cc:1403] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.006s
I20260505 07:54:05.289263 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab3850799254dda9716fd5bf9404159" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.289712 1181 raft_consensus.cc:359] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.290007 1181 raft_consensus.cc:385] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.290186 1181 raft_consensus.cc:740] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.290545 1181 consensus_queue.cc:260] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.290869 1779 consensus_queue.cc:1048] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.292286 1181 ts_tablet_manager.cc:1434] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260505 07:54:05.293231 1181 tablet_bootstrap.cc:492] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.294327 1176 tablet_bootstrap.cc:654] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.299465 1181 tablet_bootstrap.cc:654] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.300014 1931 raft_consensus.cc:493] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.300192 1931 raft_consensus.cc:515] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.301640 1931 leader_election.cc:290] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.301970 1684 tablet_loader.cc:96] loaded metadata for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb])
I20260505 07:54:05.302428 1684 tablet_loader.cc:96] loaded metadata for tablet b4515a0ff8304c69b9aa030e9b14d43f (table default.Table_3621941a687641ecbcaa200aaefbe112 [id=2164fb49c5204df0ac8c95667d342af5])
I20260505 07:54:05.303138 1684 tablet_loader.cc:96] loaded metadata for tablet b5d113a872f9400c8f28d0239ca42377 (table default.Table_02afe59354994bffb24b190e9a7abf85 [id=7579207045ec4220b966af9016a8f050])
I20260505 07:54:05.304620 913 raft_consensus.cc:2468] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:05.305541 718 leader_election.cc:304] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:05.305831 1931 raft_consensus.cc:2804] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.305953 1931 raft_consensus.cc:493] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.306131 1176 tablet_bootstrap.cc:492] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:05.306129 1870 consensus_queue.cc:1048] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.306350 1176 ts_tablet_manager.cc:1403] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.002s
I20260505 07:54:05.306555 1931 raft_consensus.cc:3060] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.306912 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141665fcd59048bf926acd0287f1853e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.307085 918 raft_consensus.cc:2468] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:05.307235 1176 raft_consensus.cc:359] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.307441 1176 raft_consensus.cc:385] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.307534 1176 raft_consensus.cc:740] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.307830 1176 consensus_queue.cc:260] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.308670 1176 ts_tablet_manager.cc:1434] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:05.309010 1176 tablet_bootstrap.cc:492] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:05.311218 1176 tablet_bootstrap.cc:654] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.314234 1684 tablet_loader.cc:96] loaded metadata for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_6f171a1e79d24e57acfe3aef43b24c7e [id=35a56d340665450ab03ba49b845ae78a])
I20260505 07:54:05.316926 1684 tablet_loader.cc:96] loaded metadata for tablet b8a85b65b1904bd6b793af48f71ed726 (table default.Table_a8605cf7bc3b44f39a895bfb7ad9934b [id=daf1e66deafd417787948bb18d615391])
I20260505 07:54:05.317369 718 leader_election.cc:304] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:05.317510 1172 tablet_bootstrap.cc:492] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.317807 1172 ts_tablet_manager.cc:1403] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.064s user 0.017s sys 0.005s
I20260505 07:54:05.318668 1684 tablet_loader.cc:96] loaded metadata for tablet ba2494780bdd4b6dae6fbdf71125da88 (table default.Table_063cc9f08ab54458b9e277ca65091b11 [id=4f094780dd764a01bb6ba8c7b08d6ef0])
I20260505 07:54:05.319260 1172 raft_consensus.cc:359] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.319470 1172 raft_consensus.cc:385] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.319566 1172 raft_consensus.cc:740] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.319855 1172 consensus_queue.cc:260] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.320691 1172 ts_tablet_manager.cc:1434] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260505 07:54:05.320988 1172 tablet_bootstrap.cc:492] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.321238 1684 tablet_loader.cc:96] loaded metadata for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_d01b41eea75c46339079b05376f7d496 [id=dcf4d78fda334683bf51e07167c9999a])
I20260505 07:54:05.323416 1684 tablet_loader.cc:96] loaded metadata for tablet c0f31343e5674c0eabb90da680bc34a4 (table default.Table_de416cf2bc944e20bb8b3d84e91585c5 [id=5dd5f713f7a24f85a694f2af5171dba3])
I20260505 07:54:05.323722 1172 tablet_bootstrap.cc:654] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.324105 1870 raft_consensus.cc:2804] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.324209 1870 raft_consensus.cc:493] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.326422 1172 tablet_bootstrap.cc:492] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.326469 1684 tablet_loader.cc:96] loaded metadata for tablet c3be796a788e4465a36bb5be44918be1 (table default.Table_37add05ae60c4b40b655ffe723848dfd [id=3a9a2fa1381b40608a07038149eae25e])
I20260505 07:54:05.326603 1172 ts_tablet_manager.cc:1403] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260505 07:54:05.326910 1684 tablet_loader.cc:96] loaded metadata for tablet c4dd3ece5cb34ceb82feeb67e75da0b5 (table default.Table_d4c7087457154e89b20abe713355e422 [id=7099b9d0095f4bf880cff569eeeb5d32])
I20260505 07:54:05.327562 1172 raft_consensus.cc:359] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.327852 1172 raft_consensus.cc:385] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.327957 1172 raft_consensus.cc:740] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.328156 1684 tablet_loader.cc:96] loaded metadata for tablet c822fafe72544ca99fde910c8e764755 (table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f])
I20260505 07:54:05.328208 1172 consensus_queue.cc:260] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.328876 1172 ts_tablet_manager.cc:1434] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:05.329422 1172 tablet_bootstrap.cc:492] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.330446 1684 tablet_loader.cc:96] loaded metadata for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0])
I20260505 07:54:05.331115 1172 tablet_bootstrap.cc:654] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.331916 1684 tablet_loader.cc:96] loaded metadata for tablet d37dc6df9da5401384a6fc5de868b41c (table default.Table_6c975d29ebde41ecb90fdaf8c72b2997 [id=0a01d809c30041a0a5b803341584f3ac])
I20260505 07:54:05.334496 1684 tablet_loader.cc:96] loaded metadata for tablet d442119970a345f28de029eb2139f5c1 (table default.Table_e7ad0f00a4fc456e944f70e21d505fd5 [id=d5242964630749eeb15c3c7c9b0f3686])
I20260505 07:54:05.335000 1684 tablet_loader.cc:96] loaded metadata for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022])
I20260505 07:54:05.336210 1684 tablet_loader.cc:96] loaded metadata for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_55fc26564d414d72992d4651acdbc6bb [id=77bd7e34452244009c813333c4c32ceb])
I20260505 07:54:05.339051 1684 tablet_loader.cc:96] loaded metadata for tablet d896e290167b4d0081e8e8b831c8ae2a (table default.Table_948a14e895244fb893e72073a38d1e14 [id=85d8a9438bef4555b0ada2eceffefc0d])
I20260505 07:54:05.339354 1172 tablet_bootstrap.cc:492] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.339566 1172 ts_tablet_manager.cc:1403] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.004s
I20260505 07:54:05.340567 1172 raft_consensus.cc:359] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.340726 1172 raft_consensus.cc:385] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.340814 1172 raft_consensus.cc:740] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.341293 1684 tablet_loader.cc:96] loaded metadata for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3])
I20260505 07:54:05.341543 1172 consensus_queue.cc:260] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.342006 1172 ts_tablet_manager.cc:1434] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:05.342257 1172 tablet_bootstrap.cc:492] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.342706 1684 tablet_loader.cc:96] loaded metadata for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_57b735b31fec4ba88439afddef823171 [id=037074511cd8426487cfdb5298d1ee8e])
I20260505 07:54:05.343776 1172 tablet_bootstrap.cc:654] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.343757 1580 consensus_queue.cc:1048] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
I20260505 07:54:05.344406 1684 tablet_loader.cc:96] loaded metadata for tablet df3a49b59ed6485f970074391f932482 (table default.Table_7e8b9279f3e244a99b8ba4a25e7c569f [id=306149ae47a745edabf863d5004d5012])
I20260505 07:54:05.345983 1684 tablet_loader.cc:96] loaded metadata for tablet e1a403e4708e4725899c1cd5b9fb0e8b (table default.Table_8564f8e67e6f4995b312673dd0d63f52 [id=de1506e299994d1fbe31033d301d4533])
I20260505 07:54:05.347009 1172 tablet_bootstrap.cc:492] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.347142 1172 ts_tablet_manager.cc:1403] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260505 07:54:05.347909 1172 raft_consensus.cc:359] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.348114 1172 raft_consensus.cc:385] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.348196 1172 raft_consensus.cc:740] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.348551 1684 tablet_loader.cc:96] loaded metadata for tablet e243711164f04faabbde53b0a09d90b1 (table default.Table_9c422539fda7423c820e2450a3f7c8db [id=5993b31e5350440f9715f1e416b90eb2])
I20260505 07:54:05.348481 1172 consensus_queue.cc:260] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.348932 1684 tablet_loader.cc:96] loaded metadata for tablet e278b86e7b5d444ebad8d0f6921d4c62 (table default.Table_633da5b39da34f5c88163c9936fd0212 [id=d53c653124364ea5acc357977c9db864])
I20260505 07:54:05.349056 1172 ts_tablet_manager.cc:1434] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:05.349340 1172 tablet_bootstrap.cc:492] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.350445 1684 tablet_loader.cc:96] loaded metadata for tablet e6a5f823db6143319d743d363a77ea47 (table default.Table_359f92a4ff554e3e80468261b0e86966 [id=9930805fdf364f25aea11f859b50bc71])
I20260505 07:54:05.350816 1684 tablet_loader.cc:96] loaded metadata for tablet e7887c3462f74740b314e3bec36e280d (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f])
I20260505 07:54:05.350838 1172 tablet_bootstrap.cc:654] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.352382 1684 tablet_loader.cc:96] loaded metadata for tablet e79a1da502604d05a62e27f575b8b8f4 (table default.Table_ab5be55234c742aea9421c70e2dad3bb [id=70c9d1bf26e34dd69abefcef94c785c5])
I20260505 07:54:05.353863 1684 tablet_loader.cc:96] loaded metadata for tablet ec145bdc0dab4642ba233bd0da2c436f (table default.Table_9201abe14fd54f8880602582b1f74cc8 [id=2f4387e6af20481fa79a3b8849a529cf])
I20260505 07:54:05.355337 1684 tablet_loader.cc:96] loaded metadata for tablet ed0d43d85fcd4d05abf10d415e8dafbe (table default.Table_d2caf74ce2d94a2ea94685efc828218b [id=7ce7150d6a004dcb94857240940133de])
I20260505 07:54:05.356922 1684 tablet_loader.cc:96] loaded metadata for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00])
I20260505 07:54:05.359196 1684 tablet_loader.cc:96] loaded metadata for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_8f7c0f8991b1415abce7b31407477b52 [id=d6192f5cec174951bf17af2e65d01f79])
I20260505 07:54:05.360924 1684 tablet_loader.cc:96] loaded metadata for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae])
I20260505 07:54:05.361810 1684 tablet_loader.cc:96] loaded metadata for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d])
I20260505 07:54:05.362593 1684 tablet_loader.cc:96] loaded metadata for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891])
I20260505 07:54:05.363376 1684 tablet_loader.cc:96] loaded metadata for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (table default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c [id=8a910f6a553a47ea87ec170e376982d1])
I20260505 07:54:05.364841 1684 tablet_loader.cc:96] loaded metadata for tablet fc0fd88889b54aac88c4f18c0abe7a43 (table default.Table_9cfdbaa4e1834e3fad8d09804f626d78 [id=a3258edb643a47ecb936efc0fc489743])
I20260505 07:54:05.366796 1684 tablet_loader.cc:96] loaded metadata for tablet fcd36b5a84b4442fb76e1610d8af4c07 (table default.Table_59efa091aa214f2698ba8a102874b9cb [id=774ee21b204e495da17b51fd7cb40c95])
I20260505 07:54:05.363117 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141665fcd59048bf926acd0287f1853e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.367465 1580 raft_consensus.cc:493] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.368319 1172 tablet_bootstrap.cc:492] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.368531 1580 raft_consensus.cc:515] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.369203 1580 leader_election.cc:290] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.369505 1932 consensus_queue.cc:1048] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:05.370477 1684 catalog_manager.cc:1486] Time spent T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Loading table and tablet metadata into memory: real 2.502s user 0.173s sys 0.047s
I20260505 07:54:05.370570 1684 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20260505 07:54:05.370608 1391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 3 candidate_status { last_received { term: 2 index: 591 } } ignore_live_leader: false dest_uuid: "23c57a2c927c4c54bccecfeb3a775311" is_pre_election: true
W20260505 07:54:05.370874 1391 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46446 (request call id 3) took 2122 ms (client timeout 646 ms). Trace:
W20260505 07:54:05.370986 1391 rpcz_store.cc:269] 0505 07:54:03.248482 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:03.248569 (+ 87us) service_pool.cc:224] Handling call
0505 07:54:05.370837 (+2122268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:05.371826 1378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 3 candidate_status { last_received { term: 2 index: 591 } } ignore_live_leader: false dest_uuid: "23c57a2c927c4c54bccecfeb3a775311" is_pre_election: true
W20260505 07:54:05.372054 1378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46446 (request call id 4) took 1303 ms (client timeout 646 ms). Trace:
W20260505 07:54:05.372159 1378 rpcz_store.cc:269] 0505 07:54:04.068574 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:04.068644 (+ 70us) service_pool.cc:224] Handling call
0505 07:54:05.372018 (+1303374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:05.372483 1354 rpcz_store.cc:275] Call kudu.master.MasterService.ConnectToMaster from 127.0.0.1:46452 (request call id 18) took 2500 ms. Trace:
I20260505 07:54:05.372575 1354 rpcz_store.cc:276] 0505 07:54:02.869148 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:02.871796 (+ 2648us) service_pool.cc:224] Handling call
0505 07:54:05.369567 (+2497771us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:05.372706 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2494780bdd4b6dae6fbdf71125da88" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.372897 920 raft_consensus.cc:2468] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:05.374228 991 leader_election.cc:304] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:05.374001 1055 raft_consensus.cc:2468] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:05.368433 1172 ts_tablet_manager.cc:1403] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.005s
I20260505 07:54:05.374879 1580 raft_consensus.cc:2804] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.374976 1580 raft_consensus.cc:493] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.375160 1580 raft_consensus.cc:3060] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.375859 1181 tablet_bootstrap.cc:492] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.376003 1181 ts_tablet_manager.cc:1403] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.083s user 0.006s sys 0.004s
I20260505 07:54:05.380286 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2494780bdd4b6dae6fbdf71125da88" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.380599 1779 consensus_queue.cc:1048] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260505 07:54:05.381319 1870 raft_consensus.cc:3060] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.382447 1580 raft_consensus.cc:515] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.382880 1684 catalog_manager.cc:1269] Loaded cluster ID: 9b9d7d3d45334a3e97a4e71f5c281d3c
I20260505 07:54:05.382972 1684 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20260505 07:54:05.383460 1878 raft_consensus.cc:493] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.384305 1181 raft_consensus.cc:359] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.384531 1878 raft_consensus.cc:515] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:05.384963 1181 raft_consensus.cc:385] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.386452 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2494780bdd4b6dae6fbdf71125da88" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:05.385151 1181 raft_consensus.cc:740] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
W20260505 07:54:05.386101 991 leader_election.cc:343] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 32c008bd99224a7595cecc607e5496a9
I20260505 07:54:05.387488 778 raft_consensus.cc:2468] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:05.387537 1181 consensus_queue.cc:260] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:05.387802 988 leader_election.cc:343] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 32c008bd99224a7595cecc607e5496a9
I20260505 07:54:05.387776 1931 raft_consensus.cc:515] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.387912 988 leader_election.cc:304] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.388104 1181 ts_tablet_manager.cc:1434] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260505 07:54:05.388233 1803 raft_consensus.cc:2749] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.388357 1181 tablet_bootstrap.cc:492] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.385423 1878 leader_election.cc:290] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:05.389482 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab3850799254dda9716fd5bf9404159" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:05.389670 1580 leader_election.cc:290] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:05.392576 717 leader_election.cc:343] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.394899 1779 consensus_queue.cc:1048] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.395857 1684 catalog_manager.cc:1514] Loading token signing keys...
I20260505 07:54:05.398391 1181 tablet_bootstrap.cc:654] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.406491 1870 raft_consensus.cc:515] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.409008 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141665fcd59048bf926acd0287f1853e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:05.409211 1049 raft_consensus.cc:3060] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.412846 1049 raft_consensus.cc:2468] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:05.414304 1181 tablet_bootstrap.cc:492] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.414453 1181 ts_tablet_manager.cc:1403] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.006s
I20260505 07:54:05.414885 717 leader_election.cc:304] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:05.415412 1181 raft_consensus.cc:359] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.415616 1181 raft_consensus.cc:385] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.415704 1181 raft_consensus.cc:740] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.415915 1181 consensus_queue.cc:260] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.416455 1864 raft_consensus.cc:2804] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:05.416615 1864 raft_consensus.cc:697] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:05.416873 1864 consensus_queue.cc:237] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.416281 1931 leader_election.cc:290] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:05.428946 1181 ts_tablet_manager.cc:1434] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.014s user 0.000s sys 0.002s
I20260505 07:54:05.429307 1181 tablet_bootstrap.cc:492] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.382179 1866 raft_consensus.cc:493] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.388882 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba2494780bdd4b6dae6fbdf71125da88" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:05.430811 920 raft_consensus.cc:3060] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.431155 1181 tablet_bootstrap.cc:654] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.390641 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab3850799254dda9716fd5bf9404159" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:05.434548 919 raft_consensus.cc:3060] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.408097 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "141665fcd59048bf926acd0287f1853e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:05.434877 917 raft_consensus.cc:3060] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.430809 1866 raft_consensus.cc:515] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.435839 1866 leader_election.cc:290] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:05.436158 1931 raft_consensus.cc:493] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.436697 1931 raft_consensus.cc:515] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.437714 1931 leader_election.cc:290] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.436626 1870 leader_election.cc:290] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.439566 915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12501fdf2cc5491cb1a06db490e70810" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.439740 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb7eaff8837485a8ae3ccb44c60471d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.439836 915 raft_consensus.cc:2468] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
W20260505 07:54:05.440532 854 leader_election.cc:343] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.440770 718 leader_election.cc:304] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:05.441423 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12501fdf2cc5491cb1a06db490e70810" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.441810 1172 raft_consensus.cc:359] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.441965 1172 raft_consensus.cc:385] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.442020 1172 raft_consensus.cc:740] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:54:05.442160 717 leader_election.cc:343] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.442937 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb7eaff8837485a8ae3ccb44c60471d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.443181 1932 raft_consensus.cc:2804] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.443270 1932 raft_consensus.cc:493] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.443389 1932 raft_consensus.cc:3060] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:05.443929 856 leader_election.cc:343] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.444043 856 leader_election.cc:304] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.444438 1866 raft_consensus.cc:2749] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.450490 1932 raft_consensus.cc:515] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.451201 1932 leader_election.cc:290] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.451570 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12501fdf2cc5491cb1a06db490e70810" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:05.452140 717 leader_election.cc:343] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.459074 915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12501fdf2cc5491cb1a06db490e70810" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:05.459272 915 raft_consensus.cc:3060] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.461589 920 raft_consensus.cc:2468] T ba2494780bdd4b6dae6fbdf71125da88 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:05.463133 856 consensus_peers.cc:597] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:05.463605 991 leader_election.cc:304] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.464264 1878 raft_consensus.cc:2804] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:05.464418 1878 raft_consensus.cc:697] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:05.464720 1878 consensus_queue.cc:237] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.464924 1181 tablet_bootstrap.cc:492] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.465059 1181 ts_tablet_manager.cc:1403] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.036s user 0.000s sys 0.003s
I20260505 07:54:05.465941 1181 raft_consensus.cc:359] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.466109 1181 raft_consensus.cc:385] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.466204 1181 raft_consensus.cc:740] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.466378 1181 consensus_queue.cc:260] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.466907 1181 ts_tablet_manager.cc:1434] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:05.467199 1181 tablet_bootstrap.cc:492] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.469633 1181 tablet_bootstrap.cc:654] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.470719 1866 consensus_queue.cc:1048] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.479507 1055 raft_consensus.cc:3060] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.491508 1864 consensus_queue.cc:1048] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.494930 1172 consensus_queue.cc:260] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.495993 1172 ts_tablet_manager.cc:1434] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.065s user 0.001s sys 0.001s
I20260505 07:54:05.501780 1172 tablet_bootstrap.cc:492] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.508432 1176 tablet_bootstrap.cc:492] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:05.511909 1176 ts_tablet_manager.cc:1403] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.203s user 0.000s sys 0.015s
I20260505 07:54:05.512962 1051 raft_consensus.cc:3060] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.518316 1803 raft_consensus.cc:493] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.518474 1803 raft_consensus.cc:515] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.519106 1803 leader_election.cc:290] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.519843 1909 raft_consensus.cc:493] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.519966 1909 raft_consensus.cc:515] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.520532 1909 leader_election.cc:290] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:05.521181 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f3544e74d941d286a2a0317b941ab6" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:05.521900 854 leader_election.cc:343] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.528990 1049 raft_consensus.cc:3060] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.530335 919 raft_consensus.cc:2468] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:05.535530 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f3544e74d941d286a2a0317b941ab6" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.535997 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af9def97a454f3f9b63d637db42c110" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.536300 918 raft_consensus.cc:2468] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:05.536590 856 leader_election.cc:343] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.536736 856 leader_election.cc:304] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.538630 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af9def97a454f3f9b63d637db42c110" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.538905 778 raft_consensus.cc:2468] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:05.539340 718 leader_election.cc:304] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:05.539937 1864 raft_consensus.cc:2804] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:05.540074 1864 raft_consensus.cc:697] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:05.540369 1864 consensus_queue.cc:237] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.541002 988 leader_election.cc:304] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:05.543110 1866 raft_consensus.cc:2749] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.546334 1932 raft_consensus.cc:493] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.546510 1932 raft_consensus.cc:515] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.547070 1932 leader_election.cc:290] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.547750 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba2cc218300472cb5fd6dc8cfb7811c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.548024 1050 raft_consensus.cc:2468] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:05.548414 717 leader_election.cc:304] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:05.548802 1932 raft_consensus.cc:2804] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.548915 1932 raft_consensus.cc:493] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.549037 1932 raft_consensus.cc:3060] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.551002 1684 catalog_manager.cc:6055] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Loaded TSK: 0
I20260505 07:54:05.551337 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba2cc218300472cb5fd6dc8cfb7811c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.551685 913 raft_consensus.cc:2468] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:05.558001 2104 raft_consensus.cc:2804] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.558240 2104 raft_consensus.cc:493] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.569072 2104 raft_consensus.cc:3060] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.582329 1181 tablet_bootstrap.cc:492] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.582527 1181 ts_tablet_manager.cc:1403] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.115s user 0.016s sys 0.011s
I20260505 07:54:05.583422 1181 raft_consensus.cc:359] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.583652 1181 raft_consensus.cc:385] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.583746 1181 raft_consensus.cc:740] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.584021 1181 consensus_queue.cc:260] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.584559 1181 ts_tablet_manager.cc:1434] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:05.584903 1181 tablet_bootstrap.cc:492] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.587275 1779 raft_consensus.cc:493] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.587469 1779 raft_consensus.cc:515] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.587980 1181 tablet_bootstrap.cc:654] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.588245 1779 leader_election.cc:290] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:05.589502 718 leader_election.cc:343] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 7219f9896fd94187ae714635077110af
I20260505 07:54:05.591429 1878 consensus_queue.cc:1048] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.591493 1866 raft_consensus.cc:493] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.591670 1866 raft_consensus.cc:515] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.592379 1866 leader_election.cc:290] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:05.594180 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de1a667b14fe494c8ee4be0f47a340be" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.594635 1779 consensus_queue.cc:1048] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:05.597263 856 leader_election.cc:343] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.597703 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de1a667b14fe494c8ee4be0f47a340be" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.597957 1176 raft_consensus.cc:359] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.598150 1176 raft_consensus.cc:385] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.598225 1176 raft_consensus.cc:740] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.598469 1176 consensus_queue.cc:260] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:05.598572 854 leader_election.cc:343] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.598699 854 leader_election.cc:304] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.599011 1176 ts_tablet_manager.cc:1434] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.086s user 0.000s sys 0.000s
I20260505 07:54:05.599290 1176 tablet_bootstrap.cc:492] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:05.599490 1866 raft_consensus.cc:2749] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.601120 1176 tablet_bootstrap.cc:654] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.606137 1181 tablet_bootstrap.cc:492] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.606284 1181 ts_tablet_manager.cc:1403] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20260505 07:54:05.607079 1932 raft_consensus.cc:515] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.607688 1932 leader_election.cc:290] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:05.607785 717 leader_election.cc:343] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 7219f9896fd94187ae714635077110af
I20260505 07:54:05.607939 717 leader_election.cc:304] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:05.608161 1932 raft_consensus.cc:2749] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.607983 1181 raft_consensus.cc:359] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.608390 1181 raft_consensus.cc:385] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.608470 1181 raft_consensus.cc:740] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.608644 1181 consensus_queue.cc:260] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.609136 1181 ts_tablet_manager.cc:1434] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:05.609222 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba2cc218300472cb5fd6dc8cfb7811c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:05.609386 1181 tablet_bootstrap.cc:492] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.609385 913 raft_consensus.cc:3060] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.610625 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba2cc218300472cb5fd6dc8cfb7811c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:05.610895 1181 tablet_bootstrap.cc:654] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.611209 1053 raft_consensus.cc:3060] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.611872 917 raft_consensus.cc:2468] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:05.612212 915 raft_consensus.cc:2468] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:05.614991 718 leader_election.cc:304] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:05.615295 1932 raft_consensus.cc:2804] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:05.615458 1932 raft_consensus.cc:697] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:05.615831 1932 consensus_queue.cc:237] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.622289 1172 tablet_bootstrap.cc:654] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.631974 1684 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20260505 07:54:05.632725 1181 tablet_bootstrap.cc:492] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.632927 1181 ts_tablet_manager.cc:1403] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.000s
I20260505 07:54:05.633638 1181 raft_consensus.cc:359] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.633834 1181 raft_consensus.cc:385] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.633925 1181 raft_consensus.cc:740] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.634145 1181 consensus_queue.cc:260] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.635479 1181 ts_tablet_manager.cc:1434] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:05.636616 1181 tablet_bootstrap.cc:492] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.639374 1181 tablet_bootstrap.cc:654] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.644165 1172 tablet_bootstrap.cc:492] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.644315 1172 ts_tablet_manager.cc:1403] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.143s user 0.004s sys 0.000s
I20260505 07:54:05.645061 1172 raft_consensus.cc:359] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.645221 1172 raft_consensus.cc:385] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.645269 1172 raft_consensus.cc:740] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.645452 1172 consensus_queue.cc:260] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.648767 1172 ts_tablet_manager.cc:1434] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:05.649080 1172 tablet_bootstrap.cc:492] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.651163 1176 tablet_bootstrap.cc:492] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:05.651310 1176 ts_tablet_manager.cc:1403] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.000s
I20260505 07:54:05.652081 1176 raft_consensus.cc:359] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.652256 1176 raft_consensus.cc:385] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.652346 1176 raft_consensus.cc:740] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.652554 1176 consensus_queue.cc:260] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.656886 1172 tablet_bootstrap.cc:654] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.658684 1176 ts_tablet_manager.cc:1434] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260505 07:54:05.662446 1176 tablet_bootstrap.cc:492] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:05.664638 1176 tablet_bootstrap.cc:654] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.665791 1181 tablet_bootstrap.cc:492] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.665930 1181 ts_tablet_manager.cc:1403] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.012s
I20260505 07:54:05.666621 1181 raft_consensus.cc:359] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.666841 1181 raft_consensus.cc:385] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.666925 1181 raft_consensus.cc:740] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.667132 1181 consensus_queue.cc:260] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.667654 1181 ts_tablet_manager.cc:1434] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:05.667938 1181 tablet_bootstrap.cc:492] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.674139 1181 tablet_bootstrap.cc:654] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.679032 1051 raft_consensus.cc:3060] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.687633 913 raft_consensus.cc:2468] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:05.688153 718 leader_election.cc:304] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:05.688530 1779 raft_consensus.cc:2804] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:05.688684 1779 raft_consensus.cc:697] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:05.689059 1779 consensus_queue.cc:237] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.694849 2104 raft_consensus.cc:515] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.695461 2104 leader_election.cc:290] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.709488 1354 catalog_manager.cc:5671] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } }
I20260505 07:54:05.710402 1354 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 83e62b7af9b847b7a9005e4345905c0d (table default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]). Expected version 3 got 0
I20260505 07:54:05.711077 1354 catalog_manager.cc:5671] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:05.711803 1354 catalog_manager.cc:5671] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:05.712543 1354 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]). Expected version 2 got 0
I20260505 07:54:05.712778 1354 catalog_manager.cc:5671] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } }
I20260505 07:54:05.713066 1354 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5683d7366de84653ace56f0edcd3818c (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:05.713932 1172 tablet_bootstrap.cc:492] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.714087 1172 ts_tablet_manager.cc:1403] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.002s
I20260505 07:54:05.715021 1172 raft_consensus.cc:359] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.715214 1172 raft_consensus.cc:385] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.715269 1172 raft_consensus.cc:740] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.715497 1172 consensus_queue.cc:260] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.717360 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af9def97a454f3f9b63d637db42c110" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:05.717535 781 raft_consensus.cc:3060] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.723133 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:05.723939 1172 ts_tablet_manager.cc:1434] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
W20260505 07:54:05.724268 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 0e4e5af08ddc4c77b5640b8295b236b1 consensus state has no leader
I20260505 07:54:05.724457 1172 tablet_bootstrap.cc:492] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.725623 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:05.740482 1351 master_service.cc:954] ReplaceTablet: received request to replace tablet e7887c3462f74740b314e3bec36e280d from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:05.740537 1349 master_service.cc:954] ReplaceTablet: received request to replace tablet 2d86b8b4dadf4fd2b6409a8cb70e313e from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:05.743441 1172 tablet_bootstrap.cc:654] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.746966 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4af9def97a454f3f9b63d637db42c110" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:05.747120 913 raft_consensus.cc:3060] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.752152 1909 raft_consensus.cc:493] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.752282 1909 raft_consensus.cc:515] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.752790 1909 leader_election.cc:290] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:05.753376 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "036d306d592f41cb96814aa60e30551e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:05.753901 854 leader_election.cc:343] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.754732 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "036d306d592f41cb96814aa60e30551e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.754750 1055 raft_consensus.cc:3060] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.756779 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:05.756814 1181 tablet_bootstrap.cc:492] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.757433 1181 ts_tablet_manager.cc:1403] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.090s user 0.015s sys 0.021s
W20260505 07:54:05.757413 856 leader_election.cc:343] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.757512 856 leader_election.cc:304] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.757956 1866 raft_consensus.cc:2749] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.758219 1181 raft_consensus.cc:359] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.758399 1181 raft_consensus.cc:385] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.758459 1181 raft_consensus.cc:740] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.759238 1181 consensus_queue.cc:260] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.760203 1176 tablet_bootstrap.cc:492] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:05.760409 1176 ts_tablet_manager.cc:1403] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.098s user 0.014s sys 0.004s
I20260505 07:54:05.761737 1176 raft_consensus.cc:359] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.762029 1176 raft_consensus.cc:385] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.762149 1176 raft_consensus.cc:740] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.762468 1176 consensus_queue.cc:260] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.738065 1355 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_d01b41eea75c46339079b05376f7d496" } new_table_name: "default.Table_6432df5b02b140d89f970c85cc4f1b91" modify_external_catalogs: true
I20260505 07:54:05.739053 1353 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_dfb010d3ab294c0aa6d95c417e8c6a4e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:05.743115 1357 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d" } modify_external_catalogs: true
I20260505 07:54:05.764277 1181 ts_tablet_manager.cc:1434] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.007s user 0.002s sys 0.002s
I20260505 07:54:05.764257 1357 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d" } modify_external_catalogs: true
I20260505 07:54:05.764634 1181 tablet_bootstrap.cc:492] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.742496 1348 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6f171a1e79d24e57acfe3aef43b24c7e" } new_table_name: "default.Table_d3f58b97177c4772a1b934bd1b4c3856" modify_external_catalogs: true
I20260505 07:54:05.742143 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_bbe5605245f44d8c86e46173550c552e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:05.743242 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_5f0d98b4110b4596a6de3749d10302c9" } new_table_name: "default.Table_00e64df3348c430399b8bb1afc7f44ee" modify_external_catalogs: true
I20260505 07:54:05.766422 1176 ts_tablet_manager.cc:1434] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260505 07:54:05.766737 1176 tablet_bootstrap.cc:492] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:05.739439 1352 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_11048b568f9948f08dee95b3421474cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:05.768945 1181 tablet_bootstrap.cc:654] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.769330 1176 tablet_bootstrap.cc:654] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:05.769353 1352 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11048b568f9948f08dee95b3421474cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:05.770852 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bbe5605245f44d8c86e46173550c552e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:05.771278 1353 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_dfb010d3ab294c0aa6d95c417e8c6a4e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:05.776051 1181 tablet_bootstrap.cc:492] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.776182 1181 ts_tablet_manager.cc:1403] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.002s
I20260505 07:54:05.776925 1181 raft_consensus.cc:359] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.777647 1181 raft_consensus.cc:385] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.777761 1181 raft_consensus.cc:740] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.780270 1181 consensus_queue.cc:260] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.781920 1181 ts_tablet_manager.cc:1434] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260505 07:54:05.783108 1181 tablet_bootstrap.cc:492] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.786531 781 raft_consensus.cc:2468] T 4af9def97a454f3f9b63d637db42c110 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:05.786866 1181 tablet_bootstrap.cc:654] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.786943 913 raft_consensus.cc:2468] T 4af9def97a454f3f9b63d637db42c110 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:05.787413 1322 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0e4e5af08ddc4c77b5640b8295b236b1 consensus state has no leader
I20260505 07:54:05.788050 988 leader_election.cc:304] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:05.788901 2104 raft_consensus.cc:2804] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:05.789057 2104 raft_consensus.cc:697] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:05.789286 2104 consensus_queue.cc:237] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.794193 1779 consensus_queue.cc:1048] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.798151 1181 tablet_bootstrap.cc:492] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.798321 1181 ts_tablet_manager.cc:1403] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260505 07:54:05.799219 1181 raft_consensus.cc:359] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.799386 1181 raft_consensus.cc:385] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.799465 1181 raft_consensus.cc:740] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.799714 1181 consensus_queue.cc:260] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.801674 1181 ts_tablet_manager.cc:1434] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:05.802083 1181 tablet_bootstrap.cc:492] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.803882 1172 tablet_bootstrap.cc:492] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.803910 1181 tablet_bootstrap.cc:654] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.804016 1172 ts_tablet_manager.cc:1403] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.080s user 0.005s sys 0.000s
I20260505 07:54:05.804801 1172 raft_consensus.cc:359] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.805016 1172 raft_consensus.cc:385] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.805078 1172 raft_consensus.cc:740] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.805356 1172 consensus_queue.cc:260] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.806178 1172 ts_tablet_manager.cc:1434] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:05.809605 1848 consensus_queue.cc:1048] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.810146 1181 tablet_bootstrap.cc:492] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.810266 1181 ts_tablet_manager.cc:1403] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260505 07:54:05.812352 1172 tablet_bootstrap.cc:492] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.812924 1181 raft_consensus.cc:359] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.813131 1181 raft_consensus.cc:385] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.813211 1181 raft_consensus.cc:740] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.813277 1779 consensus_queue.cc:1048] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.813413 1181 consensus_queue.cc:260] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.813948 1181 ts_tablet_manager.cc:1434] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260505 07:54:05.814214 1181 tablet_bootstrap.cc:492] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.814261 1172 tablet_bootstrap.cc:654] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.816030 1181 tablet_bootstrap.cc:654] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.824556 1848 raft_consensus.cc:493] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.824690 1848 raft_consensus.cc:515] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.825168 1848 leader_election.cc:290] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:05.826053 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d622612bdc404727a27b83769bf39738" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:05.826509 854 leader_election.cc:343] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.826828 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d622612bdc404727a27b83769bf39738" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:05.827258 856 leader_election.cc:343] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.827348 856 leader_election.cc:304] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.827723 1848 raft_consensus.cc:2749] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.835060 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:05.843138 1866 consensus_queue.cc:1048] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.847983 1172 tablet_bootstrap.cc:492] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.848110 1172 ts_tablet_manager.cc:1403] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.000s
I20260505 07:54:05.849015 1172 raft_consensus.cc:359] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.849153 1172 raft_consensus.cc:385] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.849196 1172 raft_consensus.cc:740] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.849385 1172 consensus_queue.cc:260] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.849844 1172 ts_tablet_manager.cc:1434] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:05.850118 1172 tablet_bootstrap.cc:492] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.851830 1172 tablet_bootstrap.cc:654] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.854385 2137 tablet_replica.cc:333] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:05.855401 2137 raft_consensus.cc:2243] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:05.855801 2137 pending_rounds.cc:70] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: Trying to abort 1 pending ops.
I20260505 07:54:05.855947 1172 tablet_bootstrap.cc:492] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.856057 1172 ts_tablet_manager.cc:1403] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260505 07:54:05.856266 2137 pending_rounds.cc:77] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555476745302016 op_type: NO_OP noop_request { }
I20260505 07:54:05.856534 2137 raft_consensus.cc:2889] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:05.856719 2137 raft_consensus.cc:2272] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:05.856681 1172 raft_consensus.cc:359] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.856832 1172 raft_consensus.cc:385] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.856925 1172 raft_consensus.cc:740] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.857110 1172 consensus_queue.cc:260] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.857515 1172 ts_tablet_manager.cc:1434] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:05.857741 1172 tablet_bootstrap.cc:492] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.857856 2137 ts_tablet_manager.cc:1916] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:05.859318 1172 tablet_bootstrap.cc:654] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:05.864456 717 consensus_peers.cc:597] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:05.874468 1351 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet e7887c3462f74740b314e3bec36e280d
I20260505 07:54:05.875705 1351 catalog_manager.cc:6741] ReplaceTablet: tablet e7887c3462f74740b314e3bec36e280d deleted and replaced by tablet fba1d19f902a4a0a8b581135f6611129
I20260505 07:54:05.877864 758 tablet_service.cc:1558] Processing DeleteTablet for tablet e7887c3462f74740b314e3bec36e280d with delete_type TABLET_DATA_DELETED (replaced by tablet fba1d19f902a4a0a8b581135f6611129) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:05.881057 1053 raft_consensus.cc:2468] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:05.886876 1355 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bdfadf8810f842698d96b6c13ffa1216 has no consensus state
I20260505 07:54:05.887329 1355 master_service.cc:954] ReplaceTablet: received request to replace tablet f200abef13ac42698a9b96fc8af82ff0 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:05.889858 1870 raft_consensus.cc:493] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.893483 1181 tablet_bootstrap.cc:492] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.893663 1181 ts_tablet_manager.cc:1403] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.079s user 0.005s sys 0.000s
I20260505 07:54:05.894783 1866 raft_consensus.cc:493] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.894920 1866 raft_consensus.cc:515] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.894974 1181 raft_consensus.cc:359] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.895190 1181 raft_consensus.cc:385] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.895558 1866 leader_election.cc:290] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:05.901818 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "934cea6666b1420eb12fe4510d2466fe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:05.902434 856 leader_election.cc:343] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.904721 1350 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_66f28ba637f24448816a64c7d316edee" } modify_external_catalogs: true
I20260505 07:54:05.904978 1350 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_66f28ba637f24448816a64c7d316edee" } modify_external_catalogs: true
I20260505 07:54:05.906970 1355 master_service.cc:954] ReplaceTablet: received request to replace tablet 840eebf9e7404bf7807b7e51996957e5 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:05.907618 1172 tablet_bootstrap.cc:492] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.907743 1172 ts_tablet_manager.cc:1403] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.000s
I20260505 07:54:05.908260 1353 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_948a14e895244fb893e72073a38d1e14" } modify_external_catalogs: true
I20260505 07:54:05.908449 1353 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_948a14e895244fb893e72073a38d1e14" } modify_external_catalogs: true
I20260505 07:54:05.911412 1172 raft_consensus.cc:359] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.922843 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:05.928951 1870 raft_consensus.cc:515] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.933507 1176 tablet_bootstrap.cc:492] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:05.933668 1176 ts_tablet_manager.cc:1403] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.167s user 0.003s sys 0.011s
I20260505 07:54:05.933971 1172 raft_consensus.cc:385] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.934057 1172 raft_consensus.cc:740] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.934309 1172 consensus_queue.cc:260] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.934834 1172 ts_tablet_manager.cc:1434] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.027s user 0.001s sys 0.000s
I20260505 07:54:05.934870 895 tablet_service.cc:1558] Processing DeleteTablet for tablet e7887c3462f74740b314e3bec36e280d with delete_type TABLET_DATA_DELETED (replaced by tablet fba1d19f902a4a0a8b581135f6611129) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:05.935104 1172 tablet_bootstrap.cc:492] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.935333 1866 consensus_queue.cc:1048] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.935998 1870 leader_election.cc:290] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.935909 1176 raft_consensus.cc:359] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.937162 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4258df74ea0841449846cdf1f6e00b9b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:05.937332 1172 tablet_bootstrap.cc:654] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.937319 921 raft_consensus.cc:2468] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:05.937614 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "934cea6666b1420eb12fe4510d2466fe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:05.938412 854 leader_election.cc:343] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.938659 854 leader_election.cc:304] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:05.939235 718 leader_election.cc:304] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:05.939889 1870 raft_consensus.cc:2804] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.939992 1870 raft_consensus.cc:493] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.940086 1870 raft_consensus.cc:3060] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.942806 1352 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_1e6b1ec4673c45e6b7b6c2696722d84b" } new_table_name: "default.Table_5e6bc85381a74e2491f9cb19a284db9c" modify_external_catalogs: true
I20260505 07:54:05.943372 1176 raft_consensus.cc:385] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.944279 1866 raft_consensus.cc:2749] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:05.946924 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_7edd42954bfd448fbb723812869def42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:05.947731 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7edd42954bfd448fbb723812869def42 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:05.948096 1176 raft_consensus.cc:740] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.948356 1176 consensus_queue.cc:260] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.948977 1355 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 840eebf9e7404bf7807b7e51996957e5
I20260505 07:54:05.949819 1176 ts_tablet_manager.cc:1434] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260505 07:54:05.949887 1355 catalog_manager.cc:6741] ReplaceTablet: tablet 840eebf9e7404bf7807b7e51996957e5 deleted and replaced by tablet bf0ce3e6c00541569d15c1b6ef347579
I20260505 07:54:05.950222 1355 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_0393e87d0a314ee898b11935e2f6666d" } modify_external_catalogs: true
I20260505 07:54:05.950366 1355 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_0393e87d0a314ee898b11935e2f6666d" } modify_external_catalogs: true
I20260505 07:54:05.951452 1172 tablet_bootstrap.cc:492] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.951573 1172 ts_tablet_manager.cc:1403] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20260505 07:54:05.952446 1172 raft_consensus.cc:359] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.952602 1172 raft_consensus.cc:385] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.952605 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 840eebf9e7404bf7807b7e51996957e5 with delete_type TABLET_DATA_DELETED (replaced by tablet bf0ce3e6c00541569d15c1b6ef347579) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:05.952656 1172 raft_consensus.cc:740] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.952852 1172 consensus_queue.cc:260] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.953352 1172 ts_tablet_manager.cc:1434] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:05.953601 1176 tablet_bootstrap.cc:492] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:05.953611 1172 tablet_bootstrap.cc:492] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.954958 2105 raft_consensus.cc:493] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.955125 2105 raft_consensus.cc:515] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.955698 1181 raft_consensus.cc:740] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.956465 1906 raft_consensus.cc:493] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.956605 1906 raft_consensus.cc:515] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.957167 1906 leader_election.cc:290] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:05.955806 2105 leader_election.cc:290] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.958086 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a13ca612f241fc97ea12124677a0b1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.958428 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dc6df9da5401384a6fc5de868b41c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:05.958839 856 leader_election.cc:343] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.957034 1181 consensus_queue.cc:260] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.959964 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4258df74ea0841449846cdf1f6e00b9b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:05.960106 1181 ts_tablet_manager.cc:1434] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.066s user 0.000s sys 0.003s
I20260505 07:54:05.961232 1172 tablet_bootstrap.cc:654] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.961196 2105 raft_consensus.cc:493] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.961371 781 raft_consensus.cc:2468] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:05.961374 2105 raft_consensus.cc:515] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.962023 1176 tablet_bootstrap.cc:654] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.962277 988 leader_election.cc:304] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:05.962396 2105 leader_election.cc:290] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.962672 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 840eebf9e7404bf7807b7e51996957e5 with delete_type TABLET_DATA_DELETED (replaced by tablet bf0ce3e6c00541569d15c1b6ef347579) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:05.962664 1866 raft_consensus.cc:493] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.963020 1866 raft_consensus.cc:515] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.963743 1866 leader_election.cc:290] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:05.964622 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36b33d265ba04e93993c7e0a4efea1a4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:05.965199 854 leader_election.cc:343] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.968873 1172 tablet_bootstrap.cc:492] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.969246 1181 tablet_bootstrap.cc:492] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:05.969877 1172 ts_tablet_manager.cc:1403] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.001s
I20260505 07:54:05.970703 1172 raft_consensus.cc:359] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.970928 1172 raft_consensus.cc:385] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.971000 1172 raft_consensus.cc:740] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.971233 1172 consensus_queue.cc:260] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.971752 1172 ts_tablet_manager.cc:1434] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:05.972030 1172 tablet_bootstrap.cc:492] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:05.972283 717 leader_election.cc:343] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:05.972824 2089 consensus_queue.cc:1048] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:05.973495 1181 tablet_bootstrap.cc:654] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.973654 1172 tablet_bootstrap.cc:654] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.974777 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a13ca612f241fc97ea12124677a0b1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:05.976714 2163 tablet_replica.cc:333] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:05.977012 2163 raft_consensus.cc:2243] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:05.978368 2163 raft_consensus.cc:2272] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:05.979039 2163 ts_tablet_manager.cc:1916] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:05.982149 1878 raft_consensus.cc:493] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:05.982293 1878 raft_consensus.cc:515] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.983074 1878 leader_election.cc:290] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.983557 1870 raft_consensus.cc:515] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:05.984270 1870 leader_election.cc:290] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:05.982054 1172 tablet_bootstrap.cc:492] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:05.986482 1172 ts_tablet_manager.cc:1403] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.003s
I20260505 07:54:05.988301 1172 raft_consensus.cc:359] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:05.988440 1172 raft_consensus.cc:385] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:05.988485 1172 raft_consensus.cc:740] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:05.988673 1172 consensus_queue.cc:260] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:05.989296 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e7887c3462f74740b314e3bec36e280d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7887c3462f74740b314e3bec36e280d already in progress: creating tablet
I20260505 07:54:05.989348 1172 ts_tablet_manager.cc:1434] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:05.989697 1172 tablet_bootstrap.cc:492] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:05.991319 1172 tablet_bootstrap.cc:654] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:05.993336 1181 tablet_bootstrap.cc:492] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:05.993471 1181 ts_tablet_manager.cc:1403] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.001s
I20260505 07:54:05.996850 2099 raft_consensus.cc:2804] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:05.996948 2099 raft_consensus.cc:493] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:05.997056 2099 raft_consensus.cc:3060] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:05.997882 1866 consensus_queue.cc:1048] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:06.000269 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_4ec0d26b7f5e4c25ac10858c5718448a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:06.000401 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36b33d265ba04e93993c7e0a4efea1a4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.000823 1172 tablet_bootstrap.cc:492] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:06.000934 1172 ts_tablet_manager.cc:1403] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.002s
W20260505 07:54:06.000907 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ec0d26b7f5e4c25ac10858c5718448a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.001670 1172 raft_consensus.cc:359] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.001807 1172 raft_consensus.cc:385] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.001856 1172 raft_consensus.cc:740] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.002036 1172 consensus_queue.cc:260] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.002146 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e4c92b7c3140fdb2d6a290aaf0ad55" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.002326 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ae6404dd64bd8bf4898768e94e4f1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.002472 781 raft_consensus.cc:2393] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:06.002526 1172 ts_tablet_manager.cc:1434] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:06.002768 1172 tablet_bootstrap.cc:492] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:06.003269 854 leader_election.cc:343] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.003377 854 leader_election.cc:304] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.003556 1866 raft_consensus.cc:2749] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:06.004367 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4258df74ea0841449846cdf1f6e00b9b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:06.004536 921 raft_consensus.cc:3060] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.004518 1181 raft_consensus.cc:359] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.004747 1181 raft_consensus.cc:385] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.004863 1181 raft_consensus.cc:740] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.005100 1181 consensus_queue.cc:260] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.005776 1181 ts_tablet_manager.cc:1434] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.012s user 0.001s sys 0.000s
I20260505 07:54:06.006095 1181 tablet_bootstrap.cc:492] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.007754 1172 tablet_bootstrap.cc:654] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.007797 1181 tablet_bootstrap.cc:654] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.009430 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_1f316191e5124266935010a5cdcc7a18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.010159 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f316191e5124266935010a5cdcc7a18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.011324 1176 tablet_bootstrap.cc:492] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:06.011500 1176 ts_tablet_manager.cc:1403] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.058s user 0.002s sys 0.006s
I20260505 07:54:06.013080 1176 raft_consensus.cc:359] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.013293 1176 raft_consensus.cc:385] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.013293 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4258df74ea0841449846cdf1f6e00b9b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:06.013375 1176 raft_consensus.cc:740] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.013481 1050 raft_consensus.cc:3060] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.013746 1054 raft_consensus.cc:3060] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.017256 1354 master_service.cc:954] ReplaceTablet: received request to replace tablet 02716f62aa354d8f839266661b7f05ea from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:06.018843 1176 consensus_queue.cc:260] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.019430 1176 ts_tablet_manager.cc:1434] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260505 07:54:06.019763 1176 tablet_bootstrap.cc:492] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:06.020112 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:06.020673 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:06.020991 1866 raft_consensus.cc:493] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.021117 1866 raft_consensus.cc:515] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.021459 1176 tablet_bootstrap.cc:654] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.021740 1866 leader_election.cc:290] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:06.021755 1358 catalog_manager.cc:5671] T 70404184502a4394a737518220f2124f P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } }
I20260505 07:54:06.022753 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 42360f8ee4ff47c2af8d287f792207de (table default.Table_829cc346ac834637ab2e2198944a8550 [id=2c91868ddba84103b9a5c78201842a9e]). Expected version 1 got 0
W20260505 07:54:06.024415 856 leader_election.cc:343] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.024526 856 leader_election.cc:304] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.024881 1866 raft_consensus.cc:2749] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:06.025861 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "297537a92684425d8508f7b3b4c9d379" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.024909 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
W20260505 07:54:06.026547 856 leader_election.cc:343] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.026646 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
I20260505 07:54:06.028934 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "297537a92684425d8508f7b3b4c9d379" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.030570 1870 raft_consensus.cc:493] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.030736 1870 raft_consensus.cc:515] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.027659 1358 catalog_manager.cc:5671] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:06.031648 1870 leader_election.cc:290] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.031996 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet c822fafe72544ca99fde910c8e764755 (table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f]). Expected version 1 got 0
I20260505 07:54:06.032792 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
W20260505 07:54:06.034022 854 leader_election.cc:343] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.034078 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1feb0387bc4afead72346398dbe9e5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:06.034344 913 raft_consensus.cc:2468] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:06.034178 854 leader_election.cc:304] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.034559 2089 consensus_queue.cc:1048] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:06.034794 1866 raft_consensus.cc:2749] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:06.035741 718 leader_election.cc:304] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.036089 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 08c6ed755fc64080a2310fe9a4e0a68b (table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b]). Expected version 3 got 0
I20260505 07:54:06.036159 2089 raft_consensus.cc:2804] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:06.036273 2089 raft_consensus.cc:493] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:06.036397 2089 raft_consensus.cc:3060] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.040485 1181 tablet_bootstrap.cc:492] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.040688 1181 ts_tablet_manager.cc:1403] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.005s
I20260505 07:54:06.041725 1181 raft_consensus.cc:359] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.041981 1181 raft_consensus.cc:385] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.042330 1176 tablet_bootstrap.cc:492] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:06.042317 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1feb0387bc4afead72346398dbe9e5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.042477 1176 ts_tablet_manager.cc:1403] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.005s
I20260505 07:54:06.042467 1048 raft_consensus.cc:2468] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:06.042076 1181 raft_consensus.cc:740] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.043257 1176 raft_consensus.cc:359] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.043428 1176 raft_consensus.cc:385] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.043293 1181 consensus_queue.cc:260] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.043531 1176 raft_consensus.cc:740] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.043756 1176 consensus_queue.cc:260] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.044042 1181 ts_tablet_manager.cc:1434] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260505 07:54:06.044453 1181 tablet_bootstrap.cc:492] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.046248 1932 raft_consensus.cc:493] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.046397 1932 raft_consensus.cc:515] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.046787 1181 tablet_bootstrap.cc:654] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.046942 1932 leader_election.cc:290] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.047400 1176 ts_tablet_manager.cc:1434] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:06.047691 1176 tablet_bootstrap.cc:492] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:06.048041 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_95fae21889c84b61ad67610d209031b2 [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 1 got 0
I20260505 07:54:06.048491 1358 catalog_manager.cc:5671] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } }
I20260505 07:54:06.048998 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e278b86e7b5d444ebad8d0f6921d4c62 (table default.Table_633da5b39da34f5c88163c9936fd0212 [id=d53c653124364ea5acc357977c9db864]). Expected version 3 got 0
I20260505 07:54:06.049405 1358 catalog_manager.cc:5671] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:06.050127 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 83e62b7af9b847b7a9005e4345905c0d (table default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]). Expected version 3 got 0
I20260505 07:54:06.050452 1176 tablet_bootstrap.cc:654] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.050465 1866 raft_consensus.cc:493] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.050594 1866 raft_consensus.cc:515] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.050395 1358 catalog_manager.cc:5671] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } }
I20260505 07:54:06.051244 1866 leader_election.cc:290] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:06.051693 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (table default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c [id=8a910f6a553a47ea87ec170e376982d1]). Expected version 2 got 0
I20260505 07:54:06.051988 1779 raft_consensus.cc:493] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.052186 1779 raft_consensus.cc:515] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.052580 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:06.052881 2186 tablet.cc:1724] T 83e62b7af9b847b7a9005e4345905c0d P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:06.052881 1779 leader_election.cc:290] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.053385 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:06.053678 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]). Expected version 2 got 0
I20260505 07:54:06.054265 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5683d7366de84653ace56f0edcd3818c (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:06.054549 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]). Expected version 1 got 0
I20260505 07:54:06.054814 1172 tablet_bootstrap.cc:492] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:06.054945 1172 ts_tablet_manager.cc:1403] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.052s user 0.003s sys 0.015s
I20260505 07:54:06.055639 1172 raft_consensus.cc:359] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.055823 1172 raft_consensus.cc:385] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.055881 1172 raft_consensus.cc:740] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.055949 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21480c42b73e4bf0ab2453b085a867c0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.056087 1172 consensus_queue.cc:260] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.056602 1172 ts_tablet_manager.cc:1434] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:06.056882 1172 tablet_bootstrap.cc:492] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:06.058035 856 leader_election.cc:343] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.058892 1172 tablet_bootstrap.cc:654] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.059705 1176 tablet_bootstrap.cc:492] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:06.059887 1176 ts_tablet_manager.cc:1403] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
I20260505 07:54:06.059751 1358 catalog_manager.cc:5671] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } }
I20260505 07:54:06.060710 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c6ed755fc64080a2310fe9a4e0a68b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:06.060870 917 raft_consensus.cc:2468] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:06.060933 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d11bd19f074bbab101448b45426b7a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:06.061019 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21480c42b73e4bf0ab2453b085a867c0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.061084 918 raft_consensus.cc:2468] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:06.062013 2137 ts_tablet_manager.cc:1929] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:06.062268 2137 log.cc:1199] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/4ba2cc218300472cb5fd6dc8cfb7811c
I20260505 07:54:06.062449 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 85d485ae2d1d4d76a6a5d988cec8291a (table default.Table_394242d5ee184ae2b22cc355a2160fb9 [id=6e71d3f928fc45c191f64a945980a107]). Expected version 1 got 0
I20260505 07:54:06.062820 2137 ts_tablet_manager.cc:1950] T 4ba2cc218300472cb5fd6dc8cfb7811c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:06.063010 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 72da22d790624b07bfca6f6b6301c925 (table default.Table_201f8409684e49be8bc5cd5249606250 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1]). Expected version 1 got 0
I20260505 07:54:06.063531 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_7e65acc795454f48ad6b8d0dc5dc163d [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 1 got 0
I20260505 07:54:06.064983 2187 tablet.cc:1724] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:06.065659 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_6de81e84d3e44828a5b8026b551bdeb3 [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 1 got 0
I20260505 07:54:06.066507 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]). Expected version 2 got 0
I20260505 07:54:06.066917 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]). Expected version 2 got 0
I20260505 07:54:06.067123 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c6ed755fc64080a2310fe9a4e0a68b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.067432 1052 raft_consensus.cc:2468] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:06.068321 717 leader_election.cc:304] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.068672 1864 raft_consensus.cc:2804] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:06.068768 1864 raft_consensus.cc:493] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:06.068868 1864 raft_consensus.cc:3060] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.068914 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d11bd19f074bbab101448b45426b7a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.069195 1053 raft_consensus.cc:2468] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:06.069803 1176 raft_consensus.cc:359] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.070022 1176 raft_consensus.cc:385] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.070109 1176 raft_consensus.cc:740] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.070355 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:06.070400 1176 consensus_queue.cc:260] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.070891 1176 ts_tablet_manager.cc:1434] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20260505 07:54:06.071190 1176 tablet_bootstrap.cc:492] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:06.071175 1358 catalog_manager.cc:5671] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } }
W20260505 07:54:06.072403 854 leader_election.cc:343] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.072569 854 leader_election.cc:304] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.072842 1866 raft_consensus.cc:2749] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:06.073359 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:06.073529 1866 raft_consensus.cc:493] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.073634 1866 raft_consensus.cc:515] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.074218 1866 leader_election.cc:290] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:06.074266 1358 catalog_manager.cc:5671] T fcd36b5a84b4442fb76e1610d8af4c07 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:06.075124 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3fc8a9d9f04636b1f31bb88411c1da" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.075356 718 leader_election.cc:304] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.075698 1779 raft_consensus.cc:2804] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:06.075893 1779 raft_consensus.cc:493] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:06.076037 1779 raft_consensus.cc:3060] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.078842 2191 tablet.cc:1724] T 83e62b7af9b847b7a9005e4345905c0d P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:06.080152 2163 ts_tablet_manager.cc:1929] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:06.080271 2163 log.cc:1199] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/840eebf9e7404bf7807b7e51996957e5
I20260505 07:54:06.080682 2163 ts_tablet_manager.cc:1950] T 840eebf9e7404bf7807b7e51996957e5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
W20260505 07:54:06.080988 854 leader_election.cc:343] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.081214 1176 tablet_bootstrap.cc:654] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.082044 1181 tablet_bootstrap.cc:492] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.082278 1181 ts_tablet_manager.cc:1403] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.003s
I20260505 07:54:06.083133 1181 raft_consensus.cc:359] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.083371 1181 raft_consensus.cc:385] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.083530 1181 raft_consensus.cc:740] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.083827 1181 consensus_queue.cc:260] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.084455 1181 ts_tablet_manager.cc:1434] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:06.084793 1181 tablet_bootstrap.cc:492] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.087688 1181 tablet_bootstrap.cc:654] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.088245 1358 catalog_manager.cc:5671] T 46120e7a5dcb482f8f93fc2aa5219aff P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } }
I20260505 07:54:06.083268 1172 tablet_bootstrap.cc:492] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:06.088801 1172 ts_tablet_manager.cc:1403] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.001s
I20260505 07:54:06.089609 1172 raft_consensus.cc:359] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.089773 1172 raft_consensus.cc:385] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.089880 1172 raft_consensus.cc:740] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.089900 1866 raft_consensus.cc:493] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.089850 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:06.090041 1866 raft_consensus.cc:515] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.090070 1172 consensus_queue.cc:260] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.090183 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_268b9129e7fb4560bcbbb7e5427975fa [id=2c06ff8eed31432e87f08e9b59049652]). Expected version 1 got 0
I20260505 07:54:06.090476 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_57b735b31fec4ba88439afddef823171 [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 1 got 0
I20260505 07:54:06.090530 1866 leader_election.cc:290] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:06.090930 1172 ts_tablet_manager.cc:1434] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:06.091162 1172 tablet_bootstrap.cc:492] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:06.092073 921 raft_consensus.cc:2468] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:06.092710 1172 tablet_bootstrap.cc:654] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.093434 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:06.093820 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:06.094841 2089 raft_consensus.cc:515] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.095324 1358 catalog_manager.cc:5671] T 55d11a9975854940a0be1693508f0841 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:06.096482 2098 raft_consensus.cc:493] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:06.096776 2098 raft_consensus.cc:515] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.097617 2098 leader_election.cc:290] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.098325 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3fc8a9d9f04636b1f31bb88411c1da" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.098678 2089 leader_election.cc:290] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.099959 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:06.100234 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 54e4f7f895d943639653c8c32724baf2 (table default.Table_ceaf4d54d57542928716e3056eac2bde [id=54621fbfaf6a4b4a874207bd986b77be]). Expected version 2 got 0
I20260505 07:54:06.100452 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:06.100768 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:06.100844 1172 tablet_bootstrap.cc:492] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:06.100999 1172 ts_tablet_manager.cc:1403] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20260505 07:54:06.101096 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:06.101397 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:06.101815 1172 raft_consensus.cc:359] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.101969 1172 raft_consensus.cc:385] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.101727 1358 catalog_manager.cc:5671] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } }
I20260505 07:54:06.102026 1172 raft_consensus.cc:740] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.102229 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet accb2dedc30e494ebd22c33594cee661 (table default.Table_fa6246cc3ccd4d7082becde923f88828 [id=860ab29ce1a44ad1b2480dbdb2b40000]). Expected version 1 got 0
I20260505 07:54:06.102228 1172 consensus_queue.cc:260] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.102715 1172 ts_tablet_manager.cc:1434] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:06.102890 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79a1da502604d05a62e27f575b8b8f4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.103013 1172 tablet_bootstrap.cc:492] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:06.102967 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:06.103538 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
W20260505 07:54:06.104035 856 leader_election.cc:343] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.104182 856 leader_election.cc:304] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.104596 1172 tablet_bootstrap.cc:654] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.104632 1909 raft_consensus.cc:2749] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:06.105015 856 leader_election.cc:343] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.105794 1909 raft_consensus.cc:493] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.105965 1909 raft_consensus.cc:515] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.106633 1909 leader_election.cc:290] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:06.112884 1172 tablet_bootstrap.cc:492] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:06.113011 1172 ts_tablet_manager.cc:1403] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260505 07:54:06.113461 1181 tablet_bootstrap.cc:492] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.113657 1181 ts_tablet_manager.cc:1403] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.000s
I20260505 07:54:06.113721 1172 raft_consensus.cc:359] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.113869 1172 raft_consensus.cc:385] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.113930 1172 raft_consensus.cc:740] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.114110 1172 consensus_queue.cc:260] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.114595 1172 ts_tablet_manager.cc:1434] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:06.114897 1172 tablet_bootstrap.cc:492] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:06.114835 1181 raft_consensus.cc:359] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.116495 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d11a9975854940a0be1693508f0841" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.120570 1172 tablet_bootstrap.cc:654] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.115120 1181 raft_consensus.cc:385] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.121276 1181 raft_consensus.cc:740] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.121616 1181 consensus_queue.cc:260] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.122421 1181 ts_tablet_manager.cc:1434] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260505 07:54:06.122697 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 4ba2cc218300472cb5fd6dc8cfb7811c (table default.Table_27c52b9b060d4dd5a80799a987d5b7c9 [id=c41c8ca8836b4b4d96bc2f1129279f06]) successfully deleted
I20260505 07:54:06.122999 2137 tablet_replica.cc:333] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
W20260505 07:54:06.123041 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e7887c3462f74740b314e3bec36e280d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7887c3462f74740b314e3bec36e280d already in progress: creating tablet
I20260505 07:54:06.123188 2137 raft_consensus.cc:2243] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:06.123467 2137 raft_consensus.cc:2272] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:06.123700 1181 tablet_bootstrap.cc:492] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.124257 2137 ts_tablet_manager.cc:1916] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:06.126294 1181 tablet_bootstrap.cc:654] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.126972 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79a1da502604d05a62e27f575b8b8f4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.127357 1172 tablet_bootstrap.cc:492] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:06.127481 1172 ts_tablet_manager.cc:1403] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.004s
I20260505 07:54:06.127871 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1feb0387bc4afead72346398dbe9e5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:06.128075 921 raft_consensus.cc:3060] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.128171 1172 raft_consensus.cc:359] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.128356 1172 raft_consensus.cc:385] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.128425 1172 raft_consensus.cc:740] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.128676 1172 consensus_queue.cc:260] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.129127 1172 ts_tablet_manager.cc:1434] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:06.129449 1172 tablet_bootstrap.cc:492] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:06.130899 1172 tablet_bootstrap.cc:654] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.131932 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad8a79245ad47738d079752e51aed2b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.136405 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad8a79245ad47738d079752e51aed2b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.137434 718 leader_election.cc:304] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.137979 1870 raft_consensus.cc:2804] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:06.138140 1870 raft_consensus.cc:697] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:06.138857 1172 tablet_bootstrap.cc:492] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:06.139024 1172 ts_tablet_manager.cc:1403] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260505 07:54:06.138993 1870 consensus_queue.cc:237] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.139930 1172 raft_consensus.cc:359] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.140136 1172 raft_consensus.cc:385] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.140236 1172 raft_consensus.cc:740] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.140453 1172 consensus_queue.cc:260] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.141052 1172 ts_tablet_manager.cc:1434] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:06.141346 1172 tablet_bootstrap.cc:492] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:06.142724 854 leader_election.cc:343] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.142925 854 leader_election.cc:304] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.143226 1866 raft_consensus.cc:2749] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:06.143814 854 leader_election.cc:343] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:06.145394 856 leader_election.cc:343] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.145530 856 leader_election.cc:304] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.145941 1866 raft_consensus.cc:2749] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:06.146473 1354 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 02716f62aa354d8f839266661b7f05ea
I20260505 07:54:06.146914 2098 consensus_queue.cc:1048] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:06.147033 1354 catalog_manager.cc:6741] ReplaceTablet: tablet 02716f62aa354d8f839266661b7f05ea deleted and replaced by tablet 9f541eff399641dd81a0771186805493
I20260505 07:54:06.147079 1172 tablet_bootstrap.cc:654] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.147373 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_89c343e38c814e91b9aa1da388b52770"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:06.147511 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 02716f62aa354d8f839266661b7f05ea with delete_type TABLET_DATA_DELETED (replaced by tablet 9f541eff399641dd81a0771186805493) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:06.147955 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89c343e38c814e91b9aa1da388b52770 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.147984 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 02716f62aa354d8f839266661b7f05ea with delete_type TABLET_DATA_DELETED (replaced by tablet 9f541eff399641dd81a0771186805493) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:06.156700 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_6b6489b552be4fef93c6cace622b4253"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.157341 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6b6489b552be4fef93c6cace622b4253 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.168519 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_ca2d7147756c4590a83c26ad0af48294"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.169170 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ca2d7147756c4590a83c26ad0af48294 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.172495 1181 tablet_bootstrap.cc:492] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.174618 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1feb0387bc4afead72346398dbe9e5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:06.174889 1053 raft_consensus.cc:3060] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.172696 1181 ts_tablet_manager.cc:1403] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.001s
I20260505 07:54:06.177418 1181 raft_consensus.cc:359] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.177583 1181 raft_consensus.cc:385] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.177688 1181 raft_consensus.cc:740] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.184085 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_46550be844b4470e8239e2ba41399a68"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.184602 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_46550be844b4470e8239e2ba41399a68 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.186036 2099 raft_consensus.cc:515] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.186655 2099 leader_election.cc:290] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.192368 1354 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_b853695095d24255abdbd32d33476f81" } new_table_name: "default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff" modify_external_catalogs: true
I20260505 07:54:06.193200 1779 raft_consensus.cc:515] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.193964 1779 leader_election.cc:290] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.195878 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 840eebf9e7404bf7807b7e51996957e5 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]) successfully deleted
I20260505 07:54:06.195947 1354 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_fa6246cc3ccd4d7082becde923f88828" } modify_external_catalogs: true
W20260505 07:54:06.196280 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 02716f62aa354d8f839266661b7f05ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02716f62aa354d8f839266661b7f05ea already in progress: creating tablet
I20260505 07:54:06.196426 1181 consensus_queue.cc:260] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.196115 1354 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_fa6246cc3ccd4d7082becde923f88828" } modify_external_catalogs: true
I20260505 07:54:06.197180 1181 ts_tablet_manager.cc:1434] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.020s user 0.000s sys 0.001s
I20260505 07:54:06.197201 921 raft_consensus.cc:2468] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:06.197515 1181 tablet_bootstrap.cc:492] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.206278 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dc6df9da5401384a6fc5de868b41c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:06.206537 782 raft_consensus.cc:3060] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.208842 1181 tablet_bootstrap.cc:654] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.209573 1909 raft_consensus.cc:493] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.209741 1909 raft_consensus.cc:515] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.210304 1909 leader_election.cc:290] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:06.211548 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1602ccbae24544c4a6cf14ea073880b4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.212262 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d11bd19f074bbab101448b45426b7a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:06.212539 921 raft_consensus.cc:3060] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.216248 1047 raft_consensus.cc:3060] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.224973 1172 tablet_bootstrap.cc:492] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:06.225208 1172 ts_tablet_manager.cc:1403] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.084s user 0.007s sys 0.020s
I20260505 07:54:06.233377 718 leader_election.cc:304] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.233825 1779 raft_consensus.cc:2804] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:06.234088 1779 raft_consensus.cc:697] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:06.234579 1779 consensus_queue.cc:237] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.237193 1909 raft_consensus.cc:493] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.237432 1909 raft_consensus.cc:515] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.238603 1909 leader_election.cc:290] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:06.240104 1172 raft_consensus.cc:359] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.240372 1172 raft_consensus.cc:385] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.240458 1172 raft_consensus.cc:740] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.240736 1172 consensus_queue.cc:260] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.241242 1172 ts_tablet_manager.cc:1434] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.016s user 0.001s sys 0.001s
I20260505 07:54:06.244956 1181 tablet_bootstrap.cc:492] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.245157 1181 ts_tablet_manager.cc:1403] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.048s user 0.004s sys 0.000s
I20260505 07:54:06.245265 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3d11bd19f074bbab101448b45426b7a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:06.245458 1048 raft_consensus.cc:3060] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.245886 2186 tablet.cc:1724] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:06.246409 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1602ccbae24544c4a6cf14ea073880b4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.246572 1181 raft_consensus.cc:359] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.246832 1181 raft_consensus.cc:385] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.246920 1181 raft_consensus.cc:740] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.247141 1181 consensus_queue.cc:260] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.247716 1181 ts_tablet_manager.cc:1434] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260505 07:54:06.250662 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46274a44a06b4306b5dac43c66c85a80" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.248898 2099 consensus_queue.cc:1048] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:06.252305 1181 tablet_bootstrap.cc:492] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:54:06.253734 856 leader_election.cc:343] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.254491 1181 tablet_bootstrap.cc:654] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:06.255142 854 leader_election.cc:343] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.255497 854 leader_election.cc:304] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.255915 1909 raft_consensus.cc:2749] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:06.256403 854 leader_election.cc:343] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.258242 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46274a44a06b4306b5dac43c66c85a80" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.259795 1909 raft_consensus.cc:493] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.259974 1909 raft_consensus.cc:515] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.260751 1909 leader_election.cc:290] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:06.266371 1909 raft_consensus.cc:493] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.266520 1909 raft_consensus.cc:515] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.267230 1909 leader_election.cc:290] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:06.269444 856 leader_election.cc:343] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.269580 856 leader_election.cc:304] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.270037 1866 raft_consensus.cc:2749] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:06.271282 2099 raft_consensus.cc:493] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.271425 2099 raft_consensus.cc:515] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.272079 2099 leader_election.cc:290] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.272989 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca12a17fb014374bf7ed764a91e6985" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:06.273625 856 leader_election.cc:343] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.276443 1864 raft_consensus.cc:515] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.277096 1864 leader_election.cc:290] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.278190 1050 raft_consensus.cc:2468] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:06.278903 782 raft_consensus.cc:2468] T d37dc6df9da5401384a6fc5de868b41c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:06.286900 1864 consensus_queue.cc:1048] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:06.287923 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb7b3fded5d4def8db8d5c1e77ddcda" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.291265 1181 tablet_bootstrap.cc:492] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.291507 1181 ts_tablet_manager.cc:1403] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.002s
I20260505 07:54:06.291970 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bf3a6bc0ac42a99a4181f820e3ca87" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.292279 776 raft_consensus.cc:2468] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:06.294591 1181 raft_consensus.cc:359] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.295339 1181 raft_consensus.cc:385] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.295449 1181 raft_consensus.cc:740] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.295730 1181 consensus_queue.cc:260] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.296484 1181 ts_tablet_manager.cc:1434] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:06.296837 1181 tablet_bootstrap.cc:492] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.298493 895 tablet_service.cc:1511] Processing CreateTablet for tablet 676e2109e59749b58931d597ce1beb9c (DEFAULT_TABLE table=default.Table_fcec571b3c7140669fe8d8c983cd616f [id=f47edf0bb7a14132a37d1f820082f0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.299386 898 tablet_service.cc:1511] Processing CreateTablet for tablet 66f069982138460da3b7ae7a0b4e3833 (DEFAULT_TABLE table=default.Table_5e1ab4e85d4b46d8a5d98434da77c9a8 [id=e86701e2ef8c4075877c58927318622f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.299507 1181 tablet_bootstrap.cc:654] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.299726 758 tablet_service.cc:1511] Processing CreateTablet for tablet 676e2109e59749b58931d597ce1beb9c (DEFAULT_TABLE table=default.Table_fcec571b3c7140669fe8d8c983cd616f [id=f47edf0bb7a14132a37d1f820082f0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.300176 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676e2109e59749b58931d597ce1beb9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.304207 889 tablet_service.cc:1511] Processing CreateTablet for tablet b8a85b65b1904bd6b793af48f71ed726 (DEFAULT_TABLE table=default.Table_a8605cf7bc3b44f39a895bfb7ad9934b [id=daf1e66deafd417787948bb18d615391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.304718 882 tablet_service.cc:1511] Processing CreateTablet for tablet b12d2d2b866348d781209fa787da7d7e (DEFAULT_TABLE table=default.Table_7ed6d926da574429be8db03bde4b4221 [id=e52c3ee18ce644d2b988c2e70c6307d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.304737 751 tablet_service.cc:1511] Processing CreateTablet for tablet 66f069982138460da3b7ae7a0b4e3833 (DEFAULT_TABLE table=default.Table_5e1ab4e85d4b46d8a5d98434da77c9a8 [id=e86701e2ef8c4075877c58927318622f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.305131 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f069982138460da3b7ae7a0b4e3833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.305112 897 tablet_service.cc:1511] Processing CreateTablet for tablet 84faf2e3d91e4cc394a3411c5b6c0bc7 (DEFAULT_TABLE table=default.Table_f8b3c6c6274f468fb9f56dad70e9043c [id=c91f3cdfd7534217ba6f4c985afd18e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.309553 762 tablet_service.cc:1511] Processing CreateTablet for tablet b8a85b65b1904bd6b793af48f71ed726 (DEFAULT_TABLE table=default.Table_a8605cf7bc3b44f39a895bfb7ad9934b [id=daf1e66deafd417787948bb18d615391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.309986 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8a85b65b1904bd6b793af48f71ed726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.312369 885 tablet_service.cc:1511] Processing CreateTablet for tablet fba1d19f902a4a0a8b581135f6611129 (DEFAULT_TABLE table=default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.314451 894 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba4533c78974853b48526c465e38941 (DEFAULT_TABLE table=default.Table_bbe5605245f44d8c86e46173550c552e [id=618b6db3796e4213b89137cf012f89d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.315344 899 tablet_service.cc:1511] Processing CreateTablet for tablet 839b963b04b24cf8af8d358a69af4f39 (DEFAULT_TABLE table=default.Table_abc41dd87e7a45c1b52bf036a9dfab27 [id=eb2e83fd347c4d2583a6fdf9bac20d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.316191 750 tablet_service.cc:1511] Processing CreateTablet for tablet b12d2d2b866348d781209fa787da7d7e (DEFAULT_TABLE table=default.Table_7ed6d926da574429be8db03bde4b4221 [id=e52c3ee18ce644d2b988c2e70c6307d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.316610 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b12d2d2b866348d781209fa787da7d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.320655 757 tablet_service.cc:1511] Processing CreateTablet for tablet 84faf2e3d91e4cc394a3411c5b6c0bc7 (DEFAULT_TABLE table=default.Table_f8b3c6c6274f468fb9f56dad70e9043c [id=c91f3cdfd7534217ba6f4c985afd18e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.320974 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84faf2e3d91e4cc394a3411c5b6c0bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.321116 1181 tablet_bootstrap.cc:492] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.321274 1181 ts_tablet_manager.cc:1403] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.008s
I20260505 07:54:06.322122 1181 raft_consensus.cc:359] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.322340 1181 raft_consensus.cc:385] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.322414 1181 raft_consensus.cc:740] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.322618 1181 consensus_queue.cc:260] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.323096 1181 ts_tablet_manager.cc:1434] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:06.323377 1181 tablet_bootstrap.cc:492] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.324213 886 tablet_service.cc:1511] Processing CreateTablet for tablet 180157193a184add848ba67ca8042109 (DEFAULT_TABLE table=default.Table_6ddd9e0382d845d1b89edce006361e1a [id=ba1b026067294e0cae324fb994815400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.325337 1181 tablet_bootstrap.cc:654] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.325739 884 tablet_service.cc:1511] Processing CreateTablet for tablet 05351965bdba4ccca8d13223cd505948 (DEFAULT_TABLE table=default.Table_54f371d41b5e4a54b08bb82a834ac0af [id=abda2c3e5e9d4c64b0d08510cb3d8aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.327358 746 tablet_service.cc:1511] Processing CreateTablet for tablet fba1d19f902a4a0a8b581135f6611129 (DEFAULT_TABLE table=default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.327718 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba1d19f902a4a0a8b581135f6611129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.332002 887 tablet_service.cc:1511] Processing CreateTablet for tablet ae829daad64545e687a9528e5c38b9ed (DEFAULT_TABLE table=default.Table_06ea1b3a616b4c5aa573a1c654538879 [id=a648fb7ba63246c38e2a9f120d6d0396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.332572 901 tablet_service.cc:1511] Processing CreateTablet for tablet af8af6d1f5d9429083002991e3470f7b (DEFAULT_TABLE table=default.Table_a930977d21a04e229868a60d643ca173 [id=28164b2107de472ab15c913fbbc9840b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.335476 1181 tablet_bootstrap.cc:492] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.335677 1181 ts_tablet_manager.cc:1403] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.003s
I20260505 07:54:06.336710 1181 raft_consensus.cc:359] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.336928 1181 raft_consensus.cc:385] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.337035 1181 raft_consensus.cc:740] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.337248 1181 consensus_queue.cc:260] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.337509 891 tablet_service.cc:1511] Processing CreateTablet for tablet bdfadf8810f842698d96b6c13ffa1216 (DEFAULT_TABLE table=default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.337898 1181 ts_tablet_manager.cc:1434] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:06.338214 1181 tablet_bootstrap.cc:492] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.338876 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:06.339200 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:06.339469 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:06.341670 896 tablet_service.cc:1511] Processing CreateTablet for tablet aacf715cb07d4fe9b2c57caea594ab87 (DEFAULT_TABLE table=default.Table_043f3d0ae81c46a9a77b3264088fd1a9 [id=37b0844f588e413f99413fa009c1ef0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.345220 1181 tablet_bootstrap.cc:654] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.345356 883 tablet_service.cc:1511] Processing CreateTablet for tablet 7b8851ec03694289ae33a86f2faa4b09 (DEFAULT_TABLE table=default.Table_f99f927cdc404f66849fad37ee1f1445 [id=3637ceb2e2d8469ebc7c9f871c73c806]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.345929 890 tablet_service.cc:1511] Processing CreateTablet for tablet 898ebd1a70984cc1b9a71009b1070bfe (DEFAULT_TABLE table=default.Table_a11d31ae6a634b5693db8de72ca33cf8 [id=256562aafb76429fb120899d8af6704d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.348534 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:06.348821 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:06.349084 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:06.349365 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:06.349516 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:06.349676 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_7e65acc795454f48ad6b8d0dc5dc163d [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 1 got 0
I20260505 07:54:06.349797 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 54e4f7f895d943639653c8c32724baf2 (table default.Table_ceaf4d54d57542928716e3056eac2bde [id=54621fbfaf6a4b4a874207bd986b77be]). Expected version 2 got 0
I20260505 07:54:06.349905 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:06.350142 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:06.350642 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
I20260505 07:54:06.350988 1181 tablet_bootstrap.cc:492] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.351145 1181 ts_tablet_manager.cc:1403] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260505 07:54:06.352159 1181 raft_consensus.cc:359] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.352408 1181 raft_consensus.cc:385] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.352355 893 tablet_service.cc:1511] Processing CreateTablet for tablet d86214559d014f028acc5a9691efc9ca (DEFAULT_TABLE table=default.Table_55fc26564d414d72992d4651acdbc6bb [id=77bd7e34452244009c813333c4c32ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.352484 1181 raft_consensus.cc:740] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.352727 1181 consensus_queue.cc:260] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.353662 1909 raft_consensus.cc:493] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.353883 1909 raft_consensus.cc:515] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.354151 900 tablet_service.cc:1511] Processing CreateTablet for tablet 411025888b55405586b128d5bc0f2021 (DEFAULT_TABLE table=default.Table_a4147e267b5f45e78812bde816ae7c8d [id=0b6c6c33d8eb41dd9b351e0d4f0e8bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.354681 1909 leader_election.cc:290] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:06.356122 892 tablet_service.cc:1511] Processing CreateTablet for tablet 5006965919824adbbb23d744fb9ff335 (DEFAULT_TABLE table=default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.356257 1181 ts_tablet_manager.cc:1434] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260505 07:54:06.356276 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c822fafe72544ca99fde910c8e764755 (table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f]). Expected version 1 got 0
I20260505 07:54:06.356606 1181 tablet_bootstrap.cc:492] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.357656 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 85d485ae2d1d4d76a6a5d988cec8291a (table default.Table_394242d5ee184ae2b22cc355a2160fb9 [id=6e71d3f928fc45c191f64a945980a107]). Expected version 1 got 0
I20260505 07:54:06.358912 888 tablet_service.cc:1511] Processing CreateTablet for tablet 99850198f4d04607b52953342daa753e (DEFAULT_TABLE table=default.Table_dfb010d3ab294c0aa6d95c417e8c6a4e [id=04bbd0b5ebc54dda81ea32f720f2cb9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.359404 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 83e62b7af9b847b7a9005e4345905c0d (table default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]). Expected version 3 got 0
I20260505 07:54:06.359536 2104 raft_consensus.cc:493] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.359896 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5683d7366de84653ace56f0edcd3818c (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:06.360832 2104 raft_consensus.cc:515] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.361599 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]). Expected version 1 got 0
I20260505 07:54:06.360496 1181 tablet_bootstrap.cc:654] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.362653 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (table default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c [id=8a910f6a553a47ea87ec170e376982d1]). Expected version 2 got 0
I20260505 07:54:06.362986 2104 leader_election.cc:290] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.364369 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_6de81e84d3e44828a5b8026b551bdeb3 [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 1 got 0
I20260505 07:54:06.365043 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:06.365562 1351 catalog_manager.cc:5671] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:06.367148 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 42360f8ee4ff47c2af8d287f792207de (table default.Table_829cc346ac834637ab2e2198944a8550 [id=2c91868ddba84103b9a5c78201842a9e]). Expected version 1 got 0
I20260505 07:54:06.368347 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c6ed755fc64080a2310fe9a4e0a68b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:06.368961 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:06.369304 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]). Expected version 2 got 0
I20260505 07:54:06.368577 1049 raft_consensus.cc:3060] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.370445 2187 tablet.cc:1724] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:06.370806 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet e278b86e7b5d444ebad8d0f6921d4c62 (table default.Table_633da5b39da34f5c88163c9936fd0212 [id=d53c653124364ea5acc357977c9db864]). Expected version 3 got 0
I20260505 07:54:06.371800 915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dc6df9da5401384a6fc5de868b41c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:06.372020 915 raft_consensus.cc:2468] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:06.372365 915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc9ae6404dd64bd8bf4898768e94e4f1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
W20260505 07:54:06.373199 856 leader_election.cc:343] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.373950 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d37dc6df9da5401384a6fc5de868b41c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:06.374074 1172 tablet_bootstrap.cc:492] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:06.374128 917 raft_consensus.cc:3060] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.375346 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 72da22d790624b07bfca6f6b6301c925 (table default.Table_201f8409684e49be8bc5cd5249606250 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1]). Expected version 1 got 0
I20260505 07:54:06.375993 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bf3a6bc0ac42a99a4181f820e3ca87" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:06.376200 920 raft_consensus.cc:2468] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:06.376538 914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1001dd888ecd4166ba217506739f71e8" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:06.376760 914 raft_consensus.cc:2468] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:06.377061 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:06.377360 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:06.377926 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:06.378116 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55d11a9975854940a0be1693508f0841" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:06.378247 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]). Expected version 2 got 0
I20260505 07:54:06.378466 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 08c6ed755fc64080a2310fe9a4e0a68b (table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b]). Expected version 3 got 0
I20260505 07:54:06.378657 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676e2109e59749b58931d597ce1beb9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.378942 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]). Expected version 2 got 0
I20260505 07:54:06.379241 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_95fae21889c84b61ad67610d209031b2 [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 1 got 0
I20260505 07:54:06.380522 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08c6ed755fc64080a2310fe9a4e0a68b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:06.380676 920 raft_consensus.cc:3060] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.382920 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f069982138460da3b7ae7a0b4e3833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.387884 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8a85b65b1904bd6b793af48f71ed726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.389418 921 raft_consensus.cc:2468] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:06.389799 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:06.390226 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_268b9129e7fb4560bcbbb7e5427975fa [id=2c06ff8eed31432e87f08e9b59049652]). Expected version 1 got 0
I20260505 07:54:06.390455 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_57b735b31fec4ba88439afddef823171 [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 1 got 0
I20260505 07:54:06.390813 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:06.393039 1351 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:06.394021 1357 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet abaa596871bb4a82aefc317932e91d35
I20260505 07:54:06.394662 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e4c92b7c3140fdb2d6a290aaf0ad55" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:06.394882 1357 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_173f279e6dff4f2eae6bec3fd5d3bdf5" } new_table_name: "default.Table_03da93b15bba4cea9230ec39dfa58b5b" modify_external_catalogs: true
I20260505 07:54:06.395864 1172 tablet_bootstrap.cc:654] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.397926 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b12d2d2b866348d781209fa787da7d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.398495 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae829daad64545e687a9528e5c38b9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.399003 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d86214559d014f028acc5a9691efc9ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.399468 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b8851ec03694289ae33a86f2faa4b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.399758 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180157193a184add848ba67ca8042109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.400542 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aacf715cb07d4fe9b2c57caea594ab87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.401073 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839b963b04b24cf8af8d358a69af4f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.401556 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5006965919824adbbb23d744fb9ff335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.402030 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba4533c78974853b48526c465e38941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.402498 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99850198f4d04607b52953342daa753e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.402923 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba1d19f902a4a0a8b581135f6611129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.403326 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af8af6d1f5d9429083002991e3470f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.403452 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 8eb7eaff8837485a8ae3ccb44c60471d
I20260505 07:54:06.403769 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 898ebd1a70984cc1b9a71009b1070bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.404179 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_37117a0adb9a48248e0a53cffbf66e9c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:06.404309 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05351965bdba4ccca8d13223cd505948. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:06.404732 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_37117a0adb9a48248e0a53cffbf66e9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.404816 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84faf2e3d91e4cc394a3411c5b6c0bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.405277 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdfadf8810f842698d96b6c13ffa1216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.405746 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 411025888b55405586b128d5bc0f2021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.414608 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_94e146286fd044179da20696e871193c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.420403 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_94e146286fd044179da20696e871193c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.420935 2191 tablet.cc:1724] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:06.431658 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb7b3fded5d4def8db8d5c1e77ddcda" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.433899 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_778f98abad284a74910d0d8c8e1d9a58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:06.434157 779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b9e8eb767a4eceb583ff9b47000d58" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.436944 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1001dd888ecd4166ba217506739f71e8" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:06.437004 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_778f98abad284a74910d0d8c8e1d9a58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:06.443372 854 leader_election.cc:343] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.443610 854 leader_election.cc:304] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.443936 1909 raft_consensus.cc:2749] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:06.444479 854 leader_election.cc:343] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.445154 1909 consensus_queue.cc:1048] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:06.446200 1172 tablet_bootstrap.cc:492] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:06.446404 1172 ts_tablet_manager.cc:1403] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.072s user 0.006s sys 0.002s
I20260505 07:54:06.447292 1172 raft_consensus.cc:359] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.447332 1350 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6de81e84d3e44828a5b8026b551bdeb3" } new_table_name: "default.Table_0bbafa19896846f1abb669a5110cfe5e" modify_external_catalogs: true
I20260505 07:54:06.447468 1172 raft_consensus.cc:385] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.447570 1172 raft_consensus.cc:740] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.448014 1172 consensus_queue.cc:260] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.448724 1172 ts_tablet_manager.cc:1434] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:06.449503 1172 tablet_bootstrap.cc:492] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:06.449564 1048 raft_consensus.cc:2468] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:06.449899 1053 raft_consensus.cc:2468] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:06.451828 1172 tablet_bootstrap.cc:654] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.453323 1181 tablet_bootstrap.cc:492] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.453610 1181 ts_tablet_manager.cc:1403] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.097s user 0.003s sys 0.004s
I20260505 07:54:06.435737 746 tablet_service.cc:1511] Processing CreateTablet for tablet 839b963b04b24cf8af8d358a69af4f39 (DEFAULT_TABLE table=default.Table_abc41dd87e7a45c1b52bf036a9dfab27 [id=eb2e83fd347c4d2583a6fdf9bac20d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.456542 1172 tablet_bootstrap.cc:492] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:06.456823 1172 ts_tablet_manager.cc:1403] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260505 07:54:06.458096 1172 raft_consensus.cc:359] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.458437 1172 raft_consensus.cc:385] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.458607 1172 raft_consensus.cc:740] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.458966 1172 consensus_queue.cc:260] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.459668 1172 ts_tablet_manager.cc:1434] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:06.460551 1172 tablet_bootstrap.cc:492] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:06.461720 1878 raft_consensus.cc:493] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.463074 1172 tablet_bootstrap.cc:654] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.464044 1878 raft_consensus.cc:515] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.465103 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b9e8eb767a4eceb583ff9b47000d58" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:06.469465 1348 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b6607b7428044e428af9d765c9c9b3ba consensus state has no leader
I20260505 07:54:06.469944 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_b5f852a0e6f44abea12de59734ee34aa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.470517 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5f852a0e6f44abea12de59734ee34aa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.470638 1878 leader_election.cc:290] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.470594 1181 raft_consensus.cc:359] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.470844 1181 raft_consensus.cc:385] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.471439 1181 raft_consensus.cc:740] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.471761 1181 consensus_queue.cc:260] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.473033 1181 ts_tablet_manager.cc:1434] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.019s user 0.002s sys 0.000s
W20260505 07:54:06.473155 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 68e8f9f7b30f4912af916c15f9ae25d8 consensus state has no leader
I20260505 07:54:06.473511 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_02e4c2a300de46c897a536456185a75b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.474004 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_02e4c2a300de46c897a536456185a75b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.474882 1181 tablet_bootstrap.cc:492] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:54:06.476534 1352 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21b48208e68247e78c12f0fec7ba07dc consensus state has no leader
I20260505 07:54:06.476909 1352 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_44941ff36f98454e9975794824299592"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.477330 1352 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_44941ff36f98454e9975794824299592 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.479468 1353 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet d896e290167b4d0081e8e8b831c8ae2a
I20260505 07:54:06.480124 1353 master_service.cc:954] ReplaceTablet: received request to replace tablet 85d485ae2d1d4d76a6a5d988cec8291a from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:06.480520 1354 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet accb2dedc30e494ebd22c33594cee661
I20260505 07:54:06.481192 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_6d46a7a584f44bfbb6910845bb4b53a2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.481652 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d46a7a584f44bfbb6910845bb4b53a2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:06.483974 1357 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3a73dbb979344805876efbafbf1b2985 consensus state has no leader
I20260505 07:54:06.484328 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_830cdb08572d46bda135fc765659e6d6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.484779 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_830cdb08572d46bda135fc765659e6d6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.486835 1181 tablet_bootstrap.cc:654] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.483971 1355 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet a9b7c6e9463342a5a73af12da4447f89
I20260505 07:54:06.491346 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_102bde0a26724f2bb7fbd86f0683813e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.491972 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_102bde0a26724f2bb7fbd86f0683813e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:06.493604 1350 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3462dcc6391741e59dcb5cd08527e50f consensus state has no leader
I20260505 07:54:06.493985 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_85003b58445a4d7fa3848473b7a93d08"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.496232 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85003b58445a4d7fa3848473b7a93d08 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.504900 1349 master_service.cc:954] ReplaceTablet: received request to replace tablet e6a5f823db6143319d743d363a77ea47 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:06.513757 782 raft_consensus.cc:2468] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:06.516932 762 tablet_service.cc:1511] Processing CreateTablet for tablet 05351965bdba4ccca8d13223cd505948 (DEFAULT_TABLE table=default.Table_54f371d41b5e4a54b08bb82a834ac0af [id=abda2c3e5e9d4c64b0d08510cb3d8aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.518959 757 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba4533c78974853b48526c465e38941 (DEFAULT_TABLE table=default.Table_bbe5605245f44d8c86e46173550c552e [id=618b6db3796e4213b89137cf012f89d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.519351 747 tablet_service.cc:1511] Processing CreateTablet for tablet ae829daad64545e687a9528e5c38b9ed (DEFAULT_TABLE table=default.Table_06ea1b3a616b4c5aa573a1c654538879 [id=a648fb7ba63246c38e2a9f120d6d0396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.519706 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839b963b04b24cf8af8d358a69af4f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.523171 750 tablet_service.cc:1511] Processing CreateTablet for tablet 180157193a184add848ba67ca8042109 (DEFAULT_TABLE table=default.Table_6ddd9e0382d845d1b89edce006361e1a [id=ba1b026067294e0cae324fb994815400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.523689 1049 raft_consensus.cc:2468] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:06.531054 988 leader_election.cc:304] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.531654 718 leader_election.cc:304] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.532155 2099 raft_consensus.cc:2804] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:06.532310 2099 raft_consensus.cc:697] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:06.532678 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180157193a184add848ba67ca8042109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.532547 2099 consensus_queue.cc:237] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.536916 758 tablet_service.cc:1511] Processing CreateTablet for tablet af8af6d1f5d9429083002991e3470f7b (DEFAULT_TABLE table=default.Table_a930977d21a04e229868a60d643ca173 [id=28164b2107de472ab15c913fbbc9840b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.535705 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca12a17fb014374bf7ed764a91e6985" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:06.540069 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 48276.4us
I20260505 07:54:06.540344 988 leader_election.cc:304] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.540716 2098 raft_consensus.cc:2804] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:06.540802 2098 raft_consensus.cc:493] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:06.540907 2098 raft_consensus.cc:3060] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.540985 1864 raft_consensus.cc:2804] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:06.546386 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae829daad64545e687a9528e5c38b9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.547127 754 tablet_service.cc:1511] Processing CreateTablet for tablet bdfadf8810f842698d96b6c13ffa1216 (DEFAULT_TABLE table=default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.548480 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af8af6d1f5d9429083002991e3470f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.551954 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05351965bdba4ccca8d13223cd505948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.555783 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba4533c78974853b48526c465e38941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.556378 1878 raft_consensus.cc:493] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.556592 1878 raft_consensus.cc:515] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.560431 1878 leader_election.cc:290] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.563702 1818 rpc_server.cc:307] RPC server started. Bound to: 127.0.105.60:35401
I20260505 07:54:06.564062 1864 raft_consensus.cc:697] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:06.564213 1181 tablet_bootstrap.cc:492] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.564496 1181 ts_tablet_manager.cc:1403] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.091s user 0.004s sys 0.004s
I20260505 07:54:06.565332 1181 raft_consensus.cc:359] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.566694 1181 raft_consensus.cc:385] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.566807 1181 raft_consensus.cc:740] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.567044 1181 consensus_queue.cc:260] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.567342 1818 server_base.cc:1193] Dumped server information to /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data/info.pb
I20260505 07:54:06.567548 1181 ts_tablet_manager.cc:1434] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260505 07:54:06.568037 1181 tablet_bootstrap.cc:492] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.568203 420 external_mini_cluster.cc:1428] Started /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu as pid 1818
I20260505 07:54:06.569262 2255 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.105.60:35401 every 8 connection(s)
I20260505 07:54:06.569978 2251 raft_consensus.cc:493] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.570612 2251 raft_consensus.cc:515] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.572461 761 tablet_service.cc:1511] Processing CreateTablet for tablet aacf715cb07d4fe9b2c57caea594ab87 (DEFAULT_TABLE table=default.Table_043f3d0ae81c46a9a77b3264088fd1a9 [id=37b0844f588e413f99413fa009c1ef0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.567636 1864 consensus_queue.cc:237] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.579830 759 tablet_service.cc:1511] Processing CreateTablet for tablet 898ebd1a70984cc1b9a71009b1070bfe (DEFAULT_TABLE table=default.Table_a11d31ae6a634b5693db8de72ca33cf8 [id=256562aafb76429fb120899d8af6704d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.580192 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdfadf8810f842698d96b6c13ffa1216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.580430 2251 leader_election.cc:290] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.576673 1181 tablet_bootstrap.cc:654] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.581369 2098 raft_consensus.cc:515] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.582063 2098 leader_election.cc:290] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.586449 756 tablet_service.cc:1511] Processing CreateTablet for tablet 411025888b55405586b128d5bc0f2021 (DEFAULT_TABLE table=default.Table_a4147e267b5f45e78812bde816ae7c8d [id=0b6c6c33d8eb41dd9b351e0d4f0e8bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.590647 751 tablet_service.cc:1511] Processing CreateTablet for tablet d86214559d014f028acc5a9691efc9ca (DEFAULT_TABLE table=default.Table_55fc26564d414d72992d4651acdbc6bb [id=77bd7e34452244009c813333c4c32ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.595845 749 tablet_service.cc:1511] Processing CreateTablet for tablet 5006965919824adbbb23d744fb9ff335 (DEFAULT_TABLE table=default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.596443 2257 sys_catalog.cc:263] Verifying existing consensus state
I20260505 07:54:06.599218 743 tablet_service.cc:1511] Processing CreateTablet for tablet 9c4fc18f6faa488882d9bffb30793cdd (DEFAULT_TABLE table=default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.605120 752 tablet_service.cc:1511] Processing CreateTablet for tablet 73042a7ff5fb43f7a781b4f98f4f4020 (DEFAULT_TABLE table=default.Table_40605e9a72634999bc80f464976c5dac [id=c1d7934526274ef09104dbff5eaacb1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.605228 1181 tablet_bootstrap.cc:492] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.605381 1181 ts_tablet_manager.cc:1403] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.037s user 0.003s sys 0.010s
I20260505 07:54:06.606225 1181 raft_consensus.cc:359] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.606464 1181 raft_consensus.cc:385] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.606565 1181 raft_consensus.cc:740] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.606828 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Bootstrap starting.
I20260505 07:54:06.606803 1181 consensus_queue.cc:260] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.606845 760 tablet_service.cc:1511] Processing CreateTablet for tablet 9b5840ac386e47d6a1229015549f9d92 (DEFAULT_TABLE table=default.Table_194f3e5aa086410eb3bd64b872d2f8af [id=1a7d610787c44274a6036482200780cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.607321 1181 ts_tablet_manager.cc:1434] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:06.607615 1181 tablet_bootstrap.cc:492] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.609390 1181 tablet_bootstrap.cc:654] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.609913 755 tablet_service.cc:1511] Processing CreateTablet for tablet fc0fd88889b54aac88c4f18c0abe7a43 (DEFAULT_TABLE table=default.Table_9cfdbaa4e1834e3fad8d09804f626d78 [id=a3258edb643a47ecb936efc0fc489743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.612186 744 tablet_service.cc:1511] Processing CreateTablet for tablet 994dbc9aa33248ee991dcf76018fe520 (DEFAULT_TABLE table=default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.612907 753 tablet_service.cc:1511] Processing CreateTablet for tablet 212e496d564a4b5ebabb22504be76bf2 (DEFAULT_TABLE table=default.Table_7001b868854949a5b9bee0ba45c2f522 [id=978b4030e04a4e32938aa646b1e85e6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.613462 745 tablet_service.cc:1558] Processing DeleteTablet for tablet abaa596871bb4a82aefc317932e91d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:06.613883 745 tablet_service.cc:1558] Processing DeleteTablet for tablet 8eb7eaff8837485a8ae3ccb44c60471d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:06.614746 745 tablet_service.cc:1558] Processing DeleteTablet for tablet d896e290167b4d0081e8e8b831c8ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:06.615170 745 tablet_service.cc:1558] Processing DeleteTablet for tablet accb2dedc30e494ebd22c33594cee661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:06.615700 745 tablet_service.cc:1558] Processing DeleteTablet for tablet a9b7c6e9463342a5a73af12da4447f89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:06.615909 1354 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9" } modify_external_catalogs: true
I20260505 07:54:06.616132 1354 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9" } modify_external_catalogs: true
I20260505 07:54:06.616742 1352 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_394242d5ee184ae2b22cc355a2160fb9" } modify_external_catalogs: true
I20260505 07:54:06.616897 1352 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_394242d5ee184ae2b22cc355a2160fb9" } modify_external_catalogs: true
I20260505 07:54:06.619323 1181 tablet_bootstrap.cc:492] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.619506 1181 ts_tablet_manager.cc:1403] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.003s
I20260505 07:54:06.620360 1181 raft_consensus.cc:359] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.620631 1181 raft_consensus.cc:385] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.620718 1181 raft_consensus.cc:740] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.620925 1181 consensus_queue.cc:260] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.621519 1181 ts_tablet_manager.cc:1434] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:06.621805 1181 tablet_bootstrap.cc:492] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.621676 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_1a1a26d7818c4e038e590867f615d13f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.622344 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1a1a26d7818c4e038e590867f615d13f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.623415 1181 tablet_bootstrap.cc:654] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.626756 1049 raft_consensus.cc:3060] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.630425 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aacf715cb07d4fe9b2c57caea594ab87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.631074 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 994dbc9aa33248ee991dcf76018fe520. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.631491 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 212e496d564a4b5ebabb22504be76bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.631986 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 411025888b55405586b128d5bc0f2021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.632454 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc0fd88889b54aac88c4f18c0abe7a43. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:06.632421 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 88401us
I20260505 07:54:06.632896 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b5840ac386e47d6a1229015549f9d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.633347 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5006965919824adbbb23d744fb9ff335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.633760 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c4fc18f6faa488882d9bffb30793cdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.634159 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 898ebd1a70984cc1b9a71009b1070bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.634538 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73042a7ff5fb43f7a781b4f98f4f4020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.634968 988 leader_election.cc:304] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.635950 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d86214559d014f028acc5a9691efc9ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.635911 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_fbe275b2392f4ab7a874b451ad147cb3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.636626 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fbe275b2392f4ab7a874b451ad147cb3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.636821 991 leader_election.cc:304] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.636996 2250 consensus_queue.cc:1048] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:06.645903 2104 raft_consensus.cc:2804] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:06.646028 2104 raft_consensus.cc:493] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:06.646129 2104 raft_consensus.cc:3060] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.649552 2105 raft_consensus.cc:2749] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:06.649966 2105 consensus_queue.cc:1048] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:06.651253 2251 raft_consensus.cc:493] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.651451 2251 raft_consensus.cc:515] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.652067 2251 leader_election.cc:290] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.653179 2250 raft_consensus.cc:493] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.653354 2250 raft_consensus.cc:515] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.653942 2250 leader_election.cc:290] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.654270 991 leader_election.cc:304] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.671182 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_af9bc6281a614df8908865ea0d5a7db9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:06.655557 991 leader_election.cc:304] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:06.671931 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_af9bc6281a614df8908865ea0d5a7db9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.655695 2105 raft_consensus.cc:2749] T 55d11a9975854940a0be1693508f0841 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:06.671973 1348 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7e65acc795454f48ad6b8d0dc5dc163d" } new_table_name: "default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd" modify_external_catalogs: true
I20260505 07:54:06.672338 2105 raft_consensus.cc:2749] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:06.675935 2104 raft_consensus.cc:515] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.677083 2104 leader_election.cc:290] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.677801 1100 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:06.686990 1355 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7e8b9279f3e244a99b8ba4a25e7c569f" } new_table_name: "default.Table_7af623f581af498b86c8c3b1adbc6b80" modify_external_catalogs: true
W20260505 07:54:06.689175 1348 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a1175f7deec14fe39098ec89ab3bc049 consensus state has no leader
I20260505 07:54:06.689689 1348 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_ed8e2add9bbf436f812de7e3fe2ed692" } new_table_name: "default.Table_61b1474d21b44e19918a653005f920a1" modify_external_catalogs: true
I20260505 07:54:06.692415 2250 raft_consensus.cc:493] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.692605 2250 raft_consensus.cc:515] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.693306 2250 leader_election.cc:290] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.704123 2104 raft_consensus.cc:493] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.704339 2104 raft_consensus.cc:515] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:06.704487 1355 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet df3a49b59ed6485f970074391f932482 consensus state has no leader
I20260505 07:54:06.704964 1355 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_ceaf4d54d57542928716e3056eac2bde" } modify_external_catalogs: true
I20260505 07:54:06.705188 1355 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_ceaf4d54d57542928716e3056eac2bde" } modify_external_catalogs: true
I20260505 07:54:06.705085 2104 leader_election.cc:290] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:06.705961 1348 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 18e909ee93a24ea9842e95357c2ba858 consensus state has no leader
I20260505 07:54:06.706262 1348 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_06ea1b3a616b4c5aa573a1c654538879" } modify_external_catalogs: true
I20260505 07:54:06.706399 1348 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_06ea1b3a616b4c5aa573a1c654538879" } modify_external_catalogs: true
I20260505 07:54:06.707854 746 tablet_service.cc:1511] Processing CreateTablet for tablet 99850198f4d04607b52953342daa753e (DEFAULT_TABLE table=default.Table_dfb010d3ab294c0aa6d95c417e8c6a4e [id=04bbd0b5ebc54dda81ea32f720f2cb9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.712528 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet e7887c3462f74740b314e3bec36e280d with delete_type TABLET_DATA_DELETED (replaced by tablet fba1d19f902a4a0a8b581135f6611129) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.723950 1350 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_44941ff36f98454e9975794824299592" } new_table_name: "default.Table_e027e2467d834e1897181fba6338e7ef" modify_external_catalogs: true
I20260505 07:54:06.726701 750 tablet_service.cc:1511] Processing CreateTablet for tablet c0f31343e5674c0eabb90da680bc34a4 (DEFAULT_TABLE table=default.Table_de416cf2bc944e20bb8b3d84e91585c5 [id=5dd5f713f7a24f85a694f2af5171dba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.729216 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99850198f4d04607b52953342daa753e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.736613 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f31343e5674c0eabb90da680bc34a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.737494 1348 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet ae829daad64545e687a9528e5c38b9ed
I20260505 07:54:06.738505 1348 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7ed6d926da574429be8db03bde4b4221" } modify_external_catalogs: true
I20260505 07:54:06.738735 1348 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7ed6d926da574429be8db03bde4b4221" } modify_external_catalogs: true
I20260505 07:54:06.740477 2201 raft_consensus.cc:493] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.740629 2201 raft_consensus.cc:515] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.741189 2201 leader_election.cc:290] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.741973 2201 raft_consensus.cc:493] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.742115 2201 raft_consensus.cc:515] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.743022 2201 leader_election.cc:290] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.744547 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde59d52fd234343b1ca13a7d4aae6bf" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.746243 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 0baf83b155494fddb3c25144608706f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.748235 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12501fdf2cc5491cb1a06db490e70810" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.748793 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab3850799254dda9716fd5bf9404159" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.749375 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bf3a6bc0ac42a99a4181f820e3ca87" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:06.749534 776 raft_consensus.cc:3060] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.750954 1348 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet b12d2d2b866348d781209fa787da7d7e
I20260505 07:54:06.751888 1348 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_bbe5605245f44d8c86e46173550c552e" } new_table_name: "default.Table_9b4d72020d274a4792c92e9350519112" modify_external_catalogs: true
I20260505 07:54:06.753086 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.753556 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.755385 782 raft_consensus.cc:3060] T ba2494780bdd4b6dae6fbdf71125da88 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:06.758438 1348 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3ba4533c78974853b48526c465e38941 consensus state has no leader
I20260505 07:54:06.758469 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet aab3850799254dda9716fd5bf9404159 with delete_type TABLET_DATA_DELETED (replaced by tablet 5006965919824adbbb23d744fb9ff335) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.758953 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_497d4c8424b841d38672247e2b4e33f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:06.759270 2268 tablet_replica.cc:333] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:06.759550 2268 raft_consensus.cc:2243] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:06.759649 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_497d4c8424b841d38672247e2b4e33f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.759984 2268 raft_consensus.cc:2272] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:06.760605 2268 ts_tablet_manager.cc:1916] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:06.768282 758 tablet_service.cc:1511] Processing CreateTablet for tablet 7b8851ec03694289ae33a86f2faa4b09 (DEFAULT_TABLE table=default.Table_f99f927cdc404f66849fad37ee1f1445 [id=3637ceb2e2d8469ebc7c9f871c73c806]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.772768 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet ccdde3e5a13e436d9f2328831e155adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.782123 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 976359601ad74b149696dc7f555b550c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.782447 2250 raft_consensus.cc:493] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.782691 2250 raft_consensus.cc:515] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.783345 2250 leader_election.cc:290] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:06.790091 1350 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1f043cd9042f4e1180d7c0cd90713b03 has no consensus state
I20260505 07:54:06.790602 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_6bf0ac886e27494d867dd60c897ba88f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.791393 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6bf0ac886e27494d867dd60c897ba88f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:06.793023 1322 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 18e909ee93a24ea9842e95357c2ba858 consensus state has no leader
I20260505 07:54:06.793114 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_d0e52326b0e041e3941fd45dab79f2da"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.793736 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0e52326b0e041e3941fd45dab79f2da in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.804838 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_0251d9bbbee946e9a6f8757aefc090d2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.805364 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0251d9bbbee946e9a6f8757aefc090d2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.816085 1353 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 85d485ae2d1d4d76a6a5d988cec8291a
I20260505 07:54:06.816650 1353 catalog_manager.cc:6741] ReplaceTablet: tablet 85d485ae2d1d4d76a6a5d988cec8291a deleted and replaced by tablet a32eeaca77c1475f879465cb1046543b
I20260505 07:54:06.817037 1349 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet e6a5f823db6143319d743d363a77ea47
I20260505 07:54:06.816944 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_cf6508cc6f1e49d5b6971e4f032fa721"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:06.817440 1349 catalog_manager.cc:6741] ReplaceTablet: tablet e6a5f823db6143319d743d363a77ea47 deleted and replaced by tablet 90612f0afb4949f491e625c61e40d247
W20260505 07:54:06.817592 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cf6508cc6f1e49d5b6971e4f032fa721 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.829301 917 raft_consensus.cc:2468] T d37dc6df9da5401384a6fc5de868b41c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:06.830468 855 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.0.105.60:35401, user_credentials={real_user=slave}} blocked reactor thread for 47280.8us
I20260505 07:54:06.833039 898 tablet_service.cc:1511] Processing CreateTablet for tablet 9c4fc18f6faa488882d9bffb30793cdd (DEFAULT_TABLE table=default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.833495 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c4fc18f6faa488882d9bffb30793cdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.838994 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_1e40e3f38bf54ca28a64fa4b0c46a123"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.843670 854 leader_election.cc:343] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.843807 854 leader_election.cc:304] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:06.844085 854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 287753us
W20260505 07:54:06.844367 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e40e3f38bf54ca28a64fa4b0c46a123 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.847349 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb7eaff8837485a8ae3ccb44c60471d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.847429 1181 tablet_bootstrap.cc:492] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:06.847607 1181 ts_tablet_manager.cc:1403] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.226s user 0.029s sys 0.110s
W20260505 07:54:06.847904 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 45648.3us
I20260505 07:54:06.848541 1181 raft_consensus.cc:359] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.848776 1181 raft_consensus.cc:385] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.848901 1181 raft_consensus.cc:740] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.849156 1181 consensus_queue.cc:260] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.849742 1181 ts_tablet_manager.cc:1434] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:06.850045 1181 tablet_bootstrap.cc:492] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:06.852167 1181 tablet_bootstrap.cc:654] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:06.855065 1909 raft_consensus.cc:2749] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:06.859736 856 leader_election.cc:343] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.861665 893 tablet_service.cc:1511] Processing CreateTablet for tablet 73042a7ff5fb43f7a781b4f98f4f4020 (DEFAULT_TABLE table=default.Table_40605e9a72634999bc80f464976c5dac [id=c1d7934526274ef09104dbff5eaacb1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.866889 886 tablet_service.cc:1511] Processing CreateTablet for tablet 9b5840ac386e47d6a1229015549f9d92 (DEFAULT_TABLE table=default.Table_194f3e5aa086410eb3bd64b872d2f8af [id=1a7d610787c44274a6036482200780cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.867138 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "934cea6666b1420eb12fe4510d2466fe" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.869452 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b8851ec03694289ae33a86f2faa4b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.870179 717 leader_election.cc:304] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.880584 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bde59d52fd234343b1ca13a7d4aae6bf" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:06.880959 856 leader_election.cc:304] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:06.881253 1866 consensus_queue.cc:1048] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260505 07:54:06.881868 1866 raft_consensus.cc:2749] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:06.887115 782 raft_consensus.cc:2468] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:06.887452 1932 raft_consensus.cc:2804] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:06.887563 1932 raft_consensus.cc:697] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:06.890512 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb7eaff8837485a8ae3ccb44c60471d" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.890729 1049 raft_consensus.cc:2468] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:06.890956 883 tablet_service.cc:1511] Processing CreateTablet for tablet fc0fd88889b54aac88c4f18c0abe7a43 (DEFAULT_TABLE table=default.Table_9cfdbaa4e1834e3fad8d09804f626d78 [id=a3258edb643a47ecb936efc0fc489743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.891427 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5191f9a357de4607aab6c7a8b4bcccdb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:06.891630 1049 raft_consensus.cc:2468] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:06.891703 1909 raft_consensus.cc:493] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:06.892429 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b57c1c618484d5d86a846fb982a2240 with delete_type TABLET_DATA_DELETED (replaced by tablet de1a667b14fe494c8ee4be0f47a340be) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.892783 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf with delete_type TABLET_DATA_DELETED (replaced by tablet 3a73dbb979344805876efbafbf1b2985) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.893117 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 840eebf9e7404bf7807b7e51996957e5 with delete_type TABLET_DATA_DELETED (replaced by tablet bf0ce3e6c00541569d15c1b6ef347579) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.893410 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 02716f62aa354d8f839266661b7f05ea with delete_type TABLET_DATA_DELETED (replaced by tablet 9f541eff399641dd81a0771186805493) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:06.893934 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 676e2109e59749b58931d597ce1beb9c (DEFAULT_TABLE table=default.Table_fcec571b3c7140669fe8d8c983cd616f [id=f47edf0bb7a14132a37d1f820082f0b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.894313 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676e2109e59749b58931d597ce1beb9c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:06.899660 1176 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.765s user 0.019s sys 0.041s
I20260505 07:54:06.899822 2089 consensus_queue.cc:1048] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260505 07:54:06.900080 1870 consensus_queue.cc:1048] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260505 07:54:06.903410 1027 tablet_service.cc:1511] Processing CreateTablet for tablet b8a85b65b1904bd6b793af48f71ed726 (DEFAULT_TABLE table=default.Table_a8605cf7bc3b44f39a895bfb7ad9934b [id=daf1e66deafd417787948bb18d615391]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.903759 1932 consensus_queue.cc:237] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.904240 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8a85b65b1904bd6b793af48f71ed726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.906697 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73042a7ff5fb43f7a781b4f98f4f4020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.907496 901 tablet_service.cc:1511] Processing CreateTablet for tablet c0f31343e5674c0eabb90da680bc34a4 (DEFAULT_TABLE table=default.Table_de416cf2bc944e20bb8b3d84e91585c5 [id=5dd5f713f7a24f85a694f2af5171dba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.907984 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f31343e5674c0eabb90da680bc34a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.909487 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1001dd888ecd4166ba217506739f71e8" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:06.909677 778 raft_consensus.cc:3060] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.909734 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc0fd88889b54aac88c4f18c0abe7a43. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:06.910048 1176 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.818s user 0.019s sys 0.041s
I20260505 07:54:06.912853 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 66f069982138460da3b7ae7a0b4e3833 (DEFAULT_TABLE table=default.Table_5e1ab4e85d4b46d8a5d98434da77c9a8 [id=e86701e2ef8c4075877c58927318622f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.913187 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f069982138460da3b7ae7a0b4e3833. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.913616 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46274a44a06b4306b5dac43c66c85a80" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.917411 2257 log.cc:826] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Log is configured to *not* fsync() on all Append() calls
I20260505 07:54:06.923364 2105 consensus_queue.cc:1048] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:06.925524 1017 tablet_service.cc:1511] Processing CreateTablet for tablet b12d2d2b866348d781209fa787da7d7e (DEFAULT_TABLE table=default.Table_7ed6d926da574429be8db03bde4b4221 [id=e52c3ee18ce644d2b988c2e70c6307d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.926141 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b12d2d2b866348d781209fa787da7d7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.926254 2105 consensus_queue.cc:1048] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:06.926955 988 leader_election.cc:343] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.927758 988 leader_election.cc:304] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:06.928082 2249 raft_consensus.cc:2804] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:06.928182 2249 raft_consensus.cc:493] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:06.928287 2249 raft_consensus.cc:3060] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.931849 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 84faf2e3d91e4cc394a3411c5b6c0bc7 (DEFAULT_TABLE table=default.Table_f8b3c6c6274f468fb9f56dad70e9043c [id=c91f3cdfd7534217ba6f4c985afd18e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.932215 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84faf2e3d91e4cc394a3411c5b6c0bc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.932189 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 3ba4533c78974853b48526c465e38941 (DEFAULT_TABLE table=default.Table_bbe5605245f44d8c86e46173550c552e [id=618b6db3796e4213b89137cf012f89d0]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:06.932300 988 leader_election.cc:343] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:06.932561 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ba4533c78974853b48526c465e38941. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.932834 1024 tablet_service.cc:1511] Processing CreateTablet for tablet fba1d19f902a4a0a8b581135f6611129 (DEFAULT_TABLE table=default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.933195 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fba1d19f902a4a0a8b581135f6611129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.935885 1176 tablet_bootstrap.cc:492] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:06.935937 752 tablet_service.cc:1558] Processing DeleteTablet for tablet ae829daad64545e687a9528e5c38b9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:06.935971 900 consensus_meta.cc:287] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533: Time spent flushing consensus metadata: real 0.516s user 0.004s sys 0.001s
I20260505 07:54:06.936038 1176 ts_tablet_manager.cc:1403] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.865s user 0.021s sys 0.041s
I20260505 07:54:06.936352 900 tablet_service.cc:1558] Processing DeleteTablet for tablet 8eb7eaff8837485a8ae3ccb44c60471d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:06.936685 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f3544e74d941d286a2a0317b941ab6" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.936789 1909 raft_consensus.cc:515] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:06.937970 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 839b963b04b24cf8af8d358a69af4f39 (DEFAULT_TABLE table=default.Table_abc41dd87e7a45c1b52bf036a9dfab27 [id=eb2e83fd347c4d2583a6fdf9bac20d53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.938335 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 839b963b04b24cf8af8d358a69af4f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.942376 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 180157193a184add848ba67ca8042109 (DEFAULT_TABLE table=default.Table_6ddd9e0382d845d1b89edce006361e1a [id=ba1b026067294e0cae324fb994815400]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.942687 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 180157193a184add848ba67ca8042109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.947353 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 05351965bdba4ccca8d13223cd505948 (DEFAULT_TABLE table=default.Table_54f371d41b5e4a54b08bb82a834ac0af [id=abda2c3e5e9d4c64b0d08510cb3d8aed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.947811 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05351965bdba4ccca8d13223cd505948. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.952239 1022 tablet_service.cc:1511] Processing CreateTablet for tablet ae829daad64545e687a9528e5c38b9ed (DEFAULT_TABLE table=default.Table_06ea1b3a616b4c5aa573a1c654538879 [id=a648fb7ba63246c38e2a9f120d6d0396]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.952567 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae829daad64545e687a9528e5c38b9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.953333 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d622612bdc404727a27b83769bf39738" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:06.956387 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_f99f3923bf7742a6a16cb971a60c1245"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:06.957763 1350 master_service.cc:954] ReplaceTablet: received request to replace tablet e8bf49b787ef4de095448751464708a4 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:06.958410 1350 master_service.cc:954] ReplaceTablet: received request to replace tablet 62d4bc3154704ba2bfde83bca10f812d from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:06.959985 1035 tablet_service.cc:1511] Processing CreateTablet for tablet af8af6d1f5d9429083002991e3470f7b (DEFAULT_TABLE table=default.Table_a930977d21a04e229868a60d643ca173 [id=28164b2107de472ab15c913fbbc9840b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.960391 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af8af6d1f5d9429083002991e3470f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.966619 2249 raft_consensus.cc:515] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.967384 2249 leader_election.cc:290] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:06.971418 1353 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_8fef0056e563428d855e8e9f8053c0cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.972038 1353 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8fef0056e563428d855e8e9f8053c0cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:06.974202 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f99f3923bf7742a6a16cb971a60c1245 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.974864 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_6d0c4c13f95a4af2a119861b7923d94a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:06.975318 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6d0c4c13f95a4af2a119861b7923d94a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:06.978645 1051 raft_consensus.cc:3060] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.981894 1048 raft_consensus.cc:3060] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:06.983227 1909 leader_election.cc:290] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:06.984777 761 tablet_service.cc:1558] Processing DeleteTablet for tablet e6a5f823db6143319d743d363a77ea47 with delete_type TABLET_DATA_DELETED (replaced by tablet 90612f0afb4949f491e625c61e40d247) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:06.985033 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 85d485ae2d1d4d76a6a5d988cec8291a with delete_type TABLET_DATA_DELETED (replaced by tablet a32eeaca77c1475f879465cb1046543b) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:06.986438 778 raft_consensus.cc:2468] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:06.987947 1176 raft_consensus.cc:359] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.988147 1176 raft_consensus.cc:385] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:06.988551 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab3850799254dda9716fd5bf9404159" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:06.988806 917 raft_consensus.cc:2393] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:06.989034 889 tablet_service.cc:1511] Processing CreateTablet for tablet 212e496d564a4b5ebabb22504be76bf2 (DEFAULT_TABLE table=default.Table_7001b868854949a5b9bee0ba45c2f522 [id=978b4030e04a4e32938aa646b1e85e6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.990265 1176 raft_consensus.cc:740] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:06.990407 1348 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_57b735b31fec4ba88439afddef823171" } new_table_name: "default.Table_041db4f8d9684d8e97ba6def427d554d" modify_external_catalogs: true
I20260505 07:54:06.990522 1176 consensus_queue.cc:260] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:06.992283 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b5840ac386e47d6a1229015549f9d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.992636 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 212e496d564a4b5ebabb22504be76bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:06.993494 1353 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_38d26c09769b42159b828ec5bb429075" } new_table_name: "default.Table_c9015cf60d4c41b18743d55fbdcdf65e" modify_external_catalogs: true
I20260505 07:54:06.994462 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_268b9129e7fb4560bcbbb7e5427975fa" } new_table_name: "default.Table_728dc87e78954a7fbbf495d74705e382" modify_external_catalogs: true
I20260505 07:54:06.996014 1176 ts_tablet_manager.cc:1434] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.055s user 0.001s sys 0.001s
I20260505 07:54:06.999315 1032 tablet_service.cc:1511] Processing CreateTablet for tablet bdfadf8810f842698d96b6c13ffa1216 (DEFAULT_TABLE table=default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:06.999673 827 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:06.999858 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdfadf8810f842698d96b6c13ffa1216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.005185 1176 tablet_bootstrap.cc:492] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:07.005718 747 tablet_service.cc:1558] Processing DeleteTablet for tablet b12d2d2b866348d781209fa787da7d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:07.006349 2137 ts_tablet_manager.cc:1929] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:07.006480 2137 log.cc:1199] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/840eebf9e7404bf7807b7e51996957e5
I20260505 07:54:07.006646 1017 tablet_service.cc:1511] Processing CreateTablet for tablet aacf715cb07d4fe9b2c57caea594ab87 (DEFAULT_TABLE table=default.Table_043f3d0ae81c46a9a77b3264088fd1a9 [id=37b0844f588e413f99413fa009c1ef0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.007036 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aacf715cb07d4fe9b2c57caea594ab87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.008513 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb7eaff8837485a8ae3ccb44c60471d" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.008745 913 raft_consensus.cc:2468] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:07.009164 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12501fdf2cc5491cb1a06db490e70810" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.009382 921 raft_consensus.cc:2393] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:07.010943 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bf3a6bc0ac42a99a4181f820e3ca87" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.011101 917 raft_consensus.cc:3060] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.012506 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5191f9a357de4607aab6c7a8b4bcccdb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.012601 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 7b8851ec03694289ae33a86f2faa4b09 (DEFAULT_TABLE table=default.Table_f99f927cdc404f66849fad37ee1f1445 [id=3637ceb2e2d8469ebc7c9f871c73c806]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.012704 913 raft_consensus.cc:2468] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:07.006922 717 leader_election.cc:304] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:07.012984 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b8851ec03694289ae33a86f2faa4b09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.013077 1870 raft_consensus.cc:2804] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.013140 1870 raft_consensus.cc:493] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.013196 1870 raft_consensus.cc:3060] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.013600 717 leader_election.cc:304] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:07.013916 2089 raft_consensus.cc:2804] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.013978 2089 raft_consensus.cc:493] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.014030 2089 raft_consensus.cc:3060] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.015959 888 tablet_service.cc:1558] Processing DeleteTablet for tablet abaa596871bb4a82aefc317932e91d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.027544 1021 tablet_service.cc:1511] Processing CreateTablet for tablet 898ebd1a70984cc1b9a71009b1070bfe (DEFAULT_TABLE table=default.Table_a11d31ae6a634b5693db8de72ca33cf8 [id=256562aafb76429fb120899d8af6704d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.028558 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 898ebd1a70984cc1b9a71009b1070bfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.027865 909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1001dd888ecd4166ba217506739f71e8" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.031893 909 raft_consensus.cc:3060] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.037400 888 tablet_service.cc:1558] Processing DeleteTablet for tablet d896e290167b4d0081e8e8b831c8ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.041693 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb7eaff8837485a8ae3ccb44c60471d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.042196 890 tablet_service.cc:1511] Processing CreateTablet for tablet 994dbc9aa33248ee991dcf76018fe520 (DEFAULT_TABLE table=default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.046877 1909 raft_consensus.cc:493] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.048031 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d622612bdc404727a27b83769bf39738" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.046206 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "934cea6666b1420eb12fe4510d2466fe" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.048940 1866 raft_consensus.cc:493] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.049115 1866 raft_consensus.cc:515] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.050048 1866 leader_election.cc:290] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.050766 907 raft_consensus.cc:2468] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.050951 912 raft_consensus.cc:2468] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.051741 1031 tablet_service.cc:1511] Processing CreateTablet for tablet d86214559d014f028acc5a9691efc9ca (DEFAULT_TABLE table=default.Table_55fc26564d414d72992d4651acdbc6bb [id=77bd7e34452244009c813333c4c32ceb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.052227 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d86214559d014f028acc5a9691efc9ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.055955 916 raft_consensus.cc:2468] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.056203 1909 raft_consensus.cc:515] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.056692 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 411025888b55405586b128d5bc0f2021 (DEFAULT_TABLE table=default.Table_a4147e267b5f45e78812bde816ae7c8d [id=0b6c6c33d8eb41dd9b351e0d4f0e8bcd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.056783 1909 leader_election.cc:290] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.056902 908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46274a44a06b4306b5dac43c66c85a80" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.057051 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 411025888b55405586b128d5bc0f2021. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.057053 908 raft_consensus.cc:2468] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:07.057361 920 consensus_meta.cc:287] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533: Time spent flushing consensus metadata: real 0.677s user 0.003s sys 0.001s
I20260505 07:54:07.057441 920 raft_consensus.cc:2468] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:07.057998 895 consensus_meta.cc:287] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533: Time spent flushing consensus metadata: real 0.675s user 0.005s sys 0.001s
I20260505 07:54:07.058389 895 tablet_service.cc:1558] Processing DeleteTablet for tablet a9b7c6e9463342a5a73af12da4447f89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.058538 895 tablet_service.cc:1558] Processing DeleteTablet for tablet ae829daad64545e687a9528e5c38b9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.058645 895 tablet_service.cc:1558] Processing DeleteTablet for tablet b12d2d2b866348d781209fa787da7d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.058739 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 85d485ae2d1d4d76a6a5d988cec8291a with delete_type TABLET_DATA_DELETED (replaced by tablet a32eeaca77c1475f879465cb1046543b) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.058851 895 tablet_service.cc:1558] Processing DeleteTablet for tablet e6a5f823db6143319d743d363a77ea47 with delete_type TABLET_DATA_DELETED (replaced by tablet 90612f0afb4949f491e625c61e40d247) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:07.059136 882 consensus_meta.cc:287] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: Time spent flushing consensus metadata: real 0.638s user 0.003s sys 0.001s
I20260505 07:54:07.059931 888 tablet_service.cc:1558] Processing DeleteTablet for tablet accb2dedc30e494ebd22c33594cee661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.060457 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 994dbc9aa33248ee991dcf76018fe520. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.063395 1020 tablet_service.cc:1511] Processing CreateTablet for tablet 5006965919824adbbb23d744fb9ff335 (DEFAULT_TABLE table=default.Table_67a8ca1fae934339bdfe388762b11ab2 [id=6dad63e723444d899e014cf136aeff89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.063797 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5006965919824adbbb23d744fb9ff335. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.074653 1020 tablet_service.cc:1511] Processing CreateTablet for tablet 99850198f4d04607b52953342daa753e (DEFAULT_TABLE table=default.Table_dfb010d3ab294c0aa6d95c417e8c6a4e [id=04bbd0b5ebc54dda81ea32f720f2cb9a]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:07.075246 1172 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.519s user 0.032s sys 0.086s
W20260505 07:54:07.075340 1172 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.611s user 0.032s sys 0.086s
W20260505 07:54:07.075971 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 216221us
I20260505 07:54:07.076275 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 9c4fc18f6faa488882d9bffb30793cdd (DEFAULT_TABLE table=default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.077865 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 73042a7ff5fb43f7a781b4f98f4f4020 (DEFAULT_TABLE table=default.Table_40605e9a72634999bc80f464976c5dac [id=c1d7934526274ef09104dbff5eaacb1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.078883 1016 tablet_service.cc:1511] Processing CreateTablet for tablet 9b5840ac386e47d6a1229015549f9d92 (DEFAULT_TABLE table=default.Table_194f3e5aa086410eb3bd64b872d2f8af [id=1a7d610787c44274a6036482200780cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.079877 1019 tablet_service.cc:1511] Processing CreateTablet for tablet fc0fd88889b54aac88c4f18c0abe7a43 (DEFAULT_TABLE table=default.Table_9cfdbaa4e1834e3fad8d09804f626d78 [id=a3258edb643a47ecb936efc0fc489743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.082077 1029 tablet_service.cc:1511] Processing CreateTablet for tablet c0f31343e5674c0eabb90da680bc34a4 (DEFAULT_TABLE table=default.Table_de416cf2bc944e20bb8b3d84e91585c5 [id=5dd5f713f7a24f85a694f2af5171dba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.082463 1018 tablet_service.cc:1511] Processing CreateTablet for tablet 994dbc9aa33248ee991dcf76018fe520 (DEFAULT_TABLE table=default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.083595 2250 raft_consensus.cc:493] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.083714 2250 raft_consensus.cc:515] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.084280 2250 leader_election.cc:290] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.087553 2137 ts_tablet_manager.cc:1950] T 840eebf9e7404bf7807b7e51996957e5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:07.091141 1172 tablet_bootstrap.cc:492] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:07.091691 1172 ts_tablet_manager.cc:1403] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.631s user 0.034s sys 0.087s
I20260505 07:54:07.091944 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d442119970a345f28de029eb2139f5c1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.092897 1049 raft_consensus.cc:3060] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.093518 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a73dbb979344805876efbafbf1b2985" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.093806 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727cacd242844455915bd9963d2db19f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.095633 1172 raft_consensus.cc:359] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.096801 1172 raft_consensus.cc:385] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:07.097231 1172 raft_consensus.cc:740] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:54:07.097886 856 leader_election.cc:343] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:07.098126 856 leader_election.cc:343] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:07.098369 856 leader_election.cc:343] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.099247 1181 tablet_bootstrap.cc:492] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:07.099464 1181 ts_tablet_manager.cc:1403] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.250s user 0.019s sys 0.060s
I20260505 07:54:07.100924 1181 raft_consensus.cc:359] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.104287 1181 raft_consensus.cc:385] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:07.104430 1181 raft_consensus.cc:740] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:07.104967 1181 consensus_queue.cc:260] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.105408 2273 raft_consensus.cc:493] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.105583 2273 raft_consensus.cc:515] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.106052 2276 tablet_replica.cc:333] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:07.106299 2276 raft_consensus.cc:2243] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:07.106637 2276 raft_consensus.cc:2272] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:07.106915 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_82a1a0d4879f449d92ddc9df6749f98f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:07.107208 2276 ts_tablet_manager.cc:1916] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:07.107401 2273 leader_election.cc:290] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.101648 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 212e496d564a4b5ebabb22504be76bf2 (DEFAULT_TABLE table=default.Table_7001b868854949a5b9bee0ba45c2f522 [id=978b4030e04a4e32938aa646b1e85e6c]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:07.107470 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82a1a0d4879f449d92ddc9df6749f98f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.106220 1172 consensus_queue.cc:260] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.108672 1181 ts_tablet_manager.cc:1434] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.009s user 0.000s sys 0.003s
W20260505 07:54:07.110383 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e7887c3462f74740b314e3bec36e280d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7887c3462f74740b314e3bec36e280d already in progress: creating tablet
I20260505 07:54:07.110348 1172 ts_tablet_manager.cc:1434] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260505 07:54:07.111205 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f3544e74d941d286a2a0317b941ab6" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.111493 911 raft_consensus.cc:2468] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.111990 1172 tablet_bootstrap.cc:492] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:07.114137 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet abaa596871bb4a82aefc317932e91d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.114286 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 8eb7eaff8837485a8ae3ccb44c60471d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.114378 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet d896e290167b4d0081e8e8b831c8ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.114461 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet accb2dedc30e494ebd22c33594cee661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.114542 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet a9b7c6e9463342a5a73af12da4447f89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.114634 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet ae829daad64545e687a9528e5c38b9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.114765 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet b12d2d2b866348d781209fa787da7d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.114881 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 85d485ae2d1d4d76a6a5d988cec8291a with delete_type TABLET_DATA_DELETED (replaced by tablet a32eeaca77c1475f879465cb1046543b) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.115012 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet e6a5f823db6143319d743d363a77ea47 with delete_type TABLET_DATA_DELETED (replaced by tablet 90612f0afb4949f491e625c61e40d247) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.116190 1866 raft_consensus.cc:493] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.116308 1866 raft_consensus.cc:515] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.116904 1866 leader_election.cc:290] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.117252 1866 raft_consensus.cc:493] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.117353 1866 raft_consensus.cc:515] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.117798 1866 leader_election.cc:290] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.119272 1909 raft_consensus.cc:493] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.119377 1909 raft_consensus.cc:515] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.119843 1909 leader_election.cc:290] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.119904 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb7eaff8837485a8ae3ccb44c60471d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.120083 1909 raft_consensus.cc:493] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.120182 1909 raft_consensus.cc:515] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.120639 1909 leader_election.cc:290] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.122376 1172 tablet_bootstrap.cc:654] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:07.122464 2258 sasl_common.cc:323] Time spent creating new SASL kudu client: real 0.282s user 0.001s sys 0.000s
I20260505 07:54:07.126160 991 leader_election.cc:304] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.126808 2105 consensus_queue.cc:1048] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:07.127810 991 leader_election.cc:304] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.133522 1350 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_633da5b39da34f5c88163c9936fd0212" } modify_external_catalogs: true
I20260505 07:54:07.133759 1350 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_633da5b39da34f5c88163c9936fd0212" } modify_external_catalogs: true
I20260505 07:54:07.134831 967 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:07.127959 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe19306f7df49fdb771d21f6560e697" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.133389 1181 tablet_bootstrap.cc:492] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:07.140158 2250 raft_consensus.cc:2749] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:07.140610 991 leader_election.cc:304] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.140959 1181 tablet_bootstrap.cc:654] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:07.152150 2250 raft_consensus.cc:2749] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:07.153743 1181 tablet_bootstrap.cc:492] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:07.153926 1181 ts_tablet_manager.cc:1403] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.005s
I20260505 07:54:07.156924 1181 raft_consensus.cc:359] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.157195 1181 raft_consensus.cc:385] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:07.157296 1181 raft_consensus.cc:740] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:07.157613 1181 consensus_queue.cc:260] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.158377 1181 ts_tablet_manager.cc:1434] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:07.158776 1181 tablet_bootstrap.cc:492] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:07.159480 917 raft_consensus.cc:2468] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.165561 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c4fc18f6faa488882d9bffb30793cdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.166090 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73042a7ff5fb43f7a781b4f98f4f4020. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.166617 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b5840ac386e47d6a1229015549f9d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.167086 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc0fd88889b54aac88c4f18c0abe7a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.167505 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0f31343e5674c0eabb90da680bc34a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.167706 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 994dbc9aa33248ee991dcf76018fe520. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.169175 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99850198f4d04607b52953342daa753e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:07.181304 856 leader_election.cc:343] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.184923 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4e5af08ddc4c77b5640b8295b236b1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.185258 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29ba55a846548679cb3b86305d0d6ed" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.185566 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a403e4708e4725899c1cd5b9fb0e8b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.187747 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f121fdd001ec4ea38ccd24a6afd9e51c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.188084 1878 raft_consensus.cc:493] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:07.188221 1878 raft_consensus.cc:515] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.189009 1878 leader_election.cc:290] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:07.194037 856 leader_election.cc:343] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.202721 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 212e496d564a4b5ebabb22504be76bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.203258 2137 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 4) took 1251 ms. Trace:
I20260505 07:54:07.203372 2137 rpcz_store.cc:276] 0505 07:54:05.952280 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:05.952326 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:06.122922 (+170596us) ts_tablet_manager.cc:1151] Deleting tablet 840eebf9e7404bf7807b7e51996957e5
0505 07:54:06.122929 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:06.689421 (+566492us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:06.967976 (+278555us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.203235 (+235259us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":170207}
I20260505 07:54:07.204324 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb7eaff8837485a8ae3ccb44c60471d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:07.205255 2137 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 5) took 1057 ms. Trace:
I20260505 07:54:07.205338 2137 rpcz_store.cc:276] 0505 07:54:06.147843 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.147896 (+ 53us) service_pool.cc:224] Handling call
0505 07:54:07.203706 (+1055810us) ts_tablet_manager.cc:1151] Deleting tablet 02716f62aa354d8f839266661b7f05ea
0505 07:54:07.203717 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:07.205234 (+ 1517us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1055610}
I20260505 07:54:07.205497 1181 tablet_bootstrap.cc:654] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:07.205890 2137 tablet_replica.cc:333] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:07.208767 900 tablet_service.cc:1511] Processing CreateTablet for tablet 6f25c3e1f7e442cba8c837b568c7b53f (DEFAULT_TABLE table=default.Table_89c343e38c814e91b9aa1da388b52770 [id=91983daad68744d2b41d11f378670fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.209169 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f25c3e1f7e442cba8c837b568c7b53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.209971 1181 tablet_bootstrap.cc:492] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:07.210104 1181 ts_tablet_manager.cc:1403] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.051s user 0.003s sys 0.000s
I20260505 07:54:07.210996 1181 raft_consensus.cc:359] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.211187 1181 raft_consensus.cc:385] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:07.211246 1181 raft_consensus.cc:740] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:07.158728 991 leader_election.cc:304] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.211483 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "036d306d592f41cb96814aa60e30551e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.211675 1878 raft_consensus.cc:2804] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.211740 1878 raft_consensus.cc:493] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.211805 1878 raft_consensus.cc:3060] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:07.212141 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 53438.4us
I20260505 07:54:07.212438 991 leader_election.cc:304] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.212836 2250 raft_consensus.cc:2804] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.212915 2250 raft_consensus.cc:493] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.212977 2250 raft_consensus.cc:3060] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.213580 991 leader_election.cc:304] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.216765 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d442119970a345f28de029eb2139f5c1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:07.218129 988 leader_election.cc:343] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:07.220662 988 leader_election.cc:343] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.220919 988 leader_election.cc:304] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:07.221379 2105 raft_consensus.cc:2804] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.219087 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42360f8ee4ff47c2af8d287f792207de" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.224265 1176 tablet_bootstrap.cc:654] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:07.228168 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 213924us
W20260505 07:54:07.229682 717 consensus_peers.cc:597] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:07.231833 2201 consensus_queue.cc:1048] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:07.251904 1870 raft_consensus.cc:515] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.252574 1870 leader_election.cc:290] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.252727 2137 raft_consensus.cc:2243] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:07.253077 2137 raft_consensus.cc:2272] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:07.253561 2137 ts_tablet_manager.cc:1916] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:07.256392 2089 raft_consensus.cc:515] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.256968 2089 leader_election.cc:290] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.272239 2268 ts_tablet_manager.cc:1929] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:07.272435 2268 log.cc:1199] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/0baf83b155494fddb3c25144608706f7
I20260505 07:54:07.273020 2268 ts_tablet_manager.cc:1950] T 0baf83b155494fddb3c25144608706f7 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:07.274058 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 840eebf9e7404bf7807b7e51996957e5 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]) successfully deleted
W20260505 07:54:07.274328 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 02716f62aa354d8f839266661b7f05ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02716f62aa354d8f839266661b7f05ea already in progress: creating tablet
W20260505 07:54:07.274564 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet abaa596871bb4a82aefc317932e91d35 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abaa596871bb4a82aefc317932e91d35 already in progress: creating tablet
W20260505 07:54:07.283936 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 70369.1us
I20260505 07:54:07.284281 991 leader_election.cc:304] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.285202 2104 raft_consensus.cc:2804] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.285300 2104 raft_consensus.cc:493] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.285377 2104 raft_consensus.cc:3060] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:07.286046 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 64797us
W20260505 07:54:07.286661 988 leader_election.cc:343] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.287029 2105 raft_consensus.cc:697] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.287264 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5191f9a357de4607aab6c7a8b4bcccdb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:07.287436 1054 raft_consensus.cc:3060] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.287375 2105 consensus_queue.cc:237] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.287817 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8eb7eaff8837485a8ae3ccb44c60471d" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:07.288028 1051 raft_consensus.cc:2393] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:07.220350 1181 consensus_queue.cc:260] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.291931 1181 ts_tablet_manager.cc:1434] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.082s user 0.001s sys 0.001s
I20260505 07:54:07.292052 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:07.292171 1181 tablet_bootstrap.cc:492] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:54:07.194248 856 leader_election.cc:343] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:07.292814 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 98550us
W20260505 07:54:07.293047 856 leader_election.cc:343] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:07.293206 856 leader_election.cc:343] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.293771 1181 tablet_bootstrap.cc:654] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:07.293978 1866 raft_consensus.cc:493] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.297355 887 tablet_service.cc:1511] Processing CreateTablet for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (DEFAULT_TABLE table=default.Table_4ec0d26b7f5e4c25ac10858c5718448a [id=77cced330f69467cb7e54e309ff5a0db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.297767 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.300271 898 tablet_service.cc:1511] Processing CreateTablet for tablet bf0ce3e6c00541569d15c1b6ef347579 (DEFAULT_TABLE table=default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.300647 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf0ce3e6c00541569d15c1b6ef347579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.301041 891 tablet_service.cc:1511] Processing CreateTablet for tablet 4c8c6d524f3f44b2a0cc33d46a39d9ea (DEFAULT_TABLE table=default.Table_7edd42954bfd448fbb723812869def42 [id=1f25e0a4dfd949659a907bc290c04772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.301110 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:07.301386 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:07.301599 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:07.301887 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:07.302207 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:07.302551 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:07.302836 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:07.303045 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:07.284278 2295 raft_consensus.cc:2804] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.303212 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:07.303273 2295 raft_consensus.cc:493] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.303351 2295 raft_consensus.cc:3060] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.303399 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:07.305083 758 tablet_service.cc:1511] Processing CreateTablet for tablet 6f25c3e1f7e442cba8c837b568c7b53f (DEFAULT_TABLE table=default.Table_89c343e38c814e91b9aa1da388b52770 [id=91983daad68744d2b41d11f378670fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.305230 747 tablet_service.cc:1511] Processing CreateTablet for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (DEFAULT_TABLE table=default.Table_4ec0d26b7f5e4c25ac10858c5718448a [id=77cced330f69467cb7e54e309ff5a0db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.307317 746 tablet_service.cc:1511] Processing CreateTablet for tablet 4c8c6d524f3f44b2a0cc33d46a39d9ea (DEFAULT_TABLE table=default.Table_7edd42954bfd448fbb723812869def42 [id=1f25e0a4dfd949659a907bc290c04772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.307988 750 tablet_service.cc:1511] Processing CreateTablet for tablet bf0ce3e6c00541569d15c1b6ef347579 (DEFAULT_TABLE table=default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.308617 752 tablet_service.cc:1511] Processing CreateTablet for tablet 4466d0ed977e4803b3b484155095c271 (DEFAULT_TABLE table=default.Table_1f316191e5124266935010a5cdcc7a18 [id=88b6ba8aaba145c2b3cac834bb9a988e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.308683 2294 raft_consensus.cc:493] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.308916 2294 raft_consensus.cc:515] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.308969 2301 raft_consensus.cc:493] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.309150 2301 raft_consensus.cc:515] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.309455 2294 leader_election.cc:290] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.309674 761 tablet_service.cc:1511] Processing CreateTablet for tablet 3fece4034c3a42169fae0c9f41dce0bb (DEFAULT_TABLE table=default.Table_46550be844b4470e8239e2ba41399a68 [id=b3ed9295a3634af8a75a142cd4a30363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.310343 757 tablet_service.cc:1511] Processing CreateTablet for tablet c01ebfd9a3514db89708ecc483be53eb (DEFAULT_TABLE table=default.Table_11048b568f9948f08dee95b3421474cb [id=b72a074c426c4c339daa2d28bab69672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.310631 2302 consensus_queue.cc:1048] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:07.311012 762 tablet_service.cc:1511] Processing CreateTablet for tablet 9f541eff399641dd81a0771186805493 (DEFAULT_TABLE table=default.Table_a86abb495238445381ac0202804b4ca0 [id=4d3b183ab74a4078beb7583c9e60e2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.319018 2294 raft_consensus.cc:493] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.321007 2301 leader_election.cc:290] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.321291 2294 raft_consensus.cc:515] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.321334 888 tablet_service.cc:1511] Processing CreateTablet for tablet c01ebfd9a3514db89708ecc483be53eb (DEFAULT_TABLE table=default.Table_11048b568f9948f08dee95b3421474cb [id=b72a074c426c4c339daa2d28bab69672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.321724 882 tablet_service.cc:1511] Processing CreateTablet for tablet 9f541eff399641dd81a0771186805493 (DEFAULT_TABLE table=default.Table_a86abb495238445381ac0202804b4ca0 [id=4d3b183ab74a4078beb7583c9e60e2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.322120 895 tablet_service.cc:1511] Processing CreateTablet for tablet f2977a0a646448249d9fbe46f970d63b (DEFAULT_TABLE table=default.Table_ca2d7147756c4590a83c26ad0af48294 [id=c35d3d29588b42fc98bab9c8ebc3966b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.294147 1866 raft_consensus.cc:515] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.323463 2294 leader_election.cc:290] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.323706 2249 raft_consensus.cc:2749] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:07.323925 1866 leader_election.cc:290] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.324786 901 tablet_service.cc:1511] Processing CreateTablet for tablet 4466d0ed977e4803b3b484155095c271 (DEFAULT_TABLE table=default.Table_1f316191e5124266935010a5cdcc7a18 [id=88b6ba8aaba145c2b3cac834bb9a988e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.325203 1864 consensus_queue.cc:1048] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:07.326135 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db162dfad2448e8ac428b3d3fbe17f7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.327454 896 tablet_service.cc:1511] Processing CreateTablet for tablet 3fece4034c3a42169fae0c9f41dce0bb (DEFAULT_TABLE table=default.Table_46550be844b4470e8239e2ba41399a68 [id=b3ed9295a3634af8a75a142cd4a30363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.327945 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c8c6d524f3f44b2a0cc33d46a39d9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.332747 1878 raft_consensus.cc:515] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.333345 1878 leader_election.cc:290] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.334461 751 tablet_service.cc:1511] Processing CreateTablet for tablet f2977a0a646448249d9fbe46f970d63b (DEFAULT_TABLE table=default.Table_ca2d7147756c4590a83c26ad0af48294 [id=c35d3d29588b42fc98bab9c8ebc3966b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.335256 749 tablet_service.cc:1511] Processing CreateTablet for tablet cf77f23a2ec647faa715fb912db3a995 (DEFAULT_TABLE table=default.Table_6b6489b552be4fef93c6cace622b4253 [id=04b3940c5da14dfa91a192fb78b3558d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.335711 2250 raft_consensus.cc:515] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.336164 2250 leader_election.cc:290] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.337492 889 tablet_service.cc:1511] Processing CreateTablet for tablet cf77f23a2ec647faa715fb912db3a995 (DEFAULT_TABLE table=default.Table_6b6489b552be4fef93c6cace622b4253 [id=04b3940c5da14dfa91a192fb78b3558d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.338025 897 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53998 (request call id 7) took 1033 ms. Trace:
I20260505 07:54:07.342482 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c01ebfd9a3514db89708ecc483be53eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.343356 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f541eff399641dd81a0771186805493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.344170 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2977a0a646448249d9fbe46f970d63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.350035 1054 raft_consensus.cc:2468] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:07.349982 2104 raft_consensus.cc:515] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.350553 2104 leader_election.cc:290] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.350694 717 leader_election.cc:304] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:07.351143 2201 raft_consensus.cc:2804] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.351293 2201 raft_consensus.cc:697] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:07.351653 2268 tablet_replica.cc:333] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:07.351702 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fece4034c3a42169fae0c9f41dce0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.351568 2201 consensus_queue.cc:237] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.351847 2268 raft_consensus.cc:2243] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:07.352067 2268 raft_consensus.cc:2272] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:07.352526 2268 ts_tablet_manager.cc:1916] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:07.352504 2295 raft_consensus.cc:515] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.352975 2295 leader_election.cc:290] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.353807 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet df3a49b59ed6485f970074391f932482 (table default.Table_7af623f581af498b86c8c3b1adbc6b80 [id=306149ae47a745edabf863d5004d5012]). Expected version 1 got 0
I20260505 07:54:07.354483 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet c822fafe72544ca99fde910c8e764755 (table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f]). Expected version 1 got 0
I20260505 07:54:07.354832 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
I20260505 07:54:07.355147 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 08c6ed755fc64080a2310fe9a4e0a68b (table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b]). Expected version 3 got 0
I20260505 07:54:07.355389 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_95fae21889c84b61ad67610d209031b2 [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 1 got 0
I20260505 07:54:07.355608 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:07.355906 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:07.361450 885 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53998 (request call id 8) took 1049 ms. Trace:
I20260505 07:54:07.363135 909 raft_consensus.cc:2468] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:07.365082 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 47607.9us
W20260505 07:54:07.365310 991 leader_election.cc:343] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:54:07.366024 991 leader_election.cc:304] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.366259 2300 raft_consensus.cc:2804] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.366374 2300 raft_consensus.cc:697] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.366643 2300 consensus_queue.cc:237] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.367933 893 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53998 (request call id 24) took 1001 ms. Trace:
I20260505 07:54:07.370276 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:07.370539 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]). Expected version 2 got 0
I20260505 07:54:07.370819 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:07.371022 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:07.371202 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:07.371392 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 42360f8ee4ff47c2af8d287f792207de (table default.Table_829cc346ac834637ab2e2198944a8550 [id=2c91868ddba84103b9a5c78201842a9e]). Expected version 1 got 0
I20260505 07:54:07.371631 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 72da22d790624b07bfca6f6b6301c925 (table default.Table_201f8409684e49be8bc5cd5249606250 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1]). Expected version 1 got 0
I20260505 07:54:07.372133 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:07.372471 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:07.372797 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:07.373718 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf77f23a2ec647faa715fb912db3a995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.379913 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4466d0ed977e4803b3b484155095c271. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:07.381108 856 leader_election.cc:343] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.338099 897 rpcz_store.cc:276] 0505 07:54:06.304311 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.304946 (+ 635us) service_pool.cc:224] Handling call
0505 07:54:06.392126 (+ 87180us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0505 07:54:06.392160 (+ 34us) ts_tablet_manager.cc:682] Creating new metadata...
0505 07:54:07.014046 (+621886us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.323203 (+309157us) tablet_replica.cc:206] Creating consensus instance
0505 07:54:07.338005 (+ 14802us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:07.368031 893 rpcz_store.cc:276] 0505 07:54:06.366868 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.861415 (+494547us) service_pool.cc:224] Handling call
0505 07:54:06.861873 (+ 458us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0505 07:54:06.861901 (+ 28us) ts_tablet_manager.cc:682] Creating new metadata...
0505 07:54:07.030913 (+169012us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.367726 (+336813us) tablet_replica.cc:206] Creating consensus instance
0505 07:54:07.367899 (+ 173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:07.382901 909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "036d306d592f41cb96814aa60e30551e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.383065 909 raft_consensus.cc:2468] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.383271 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de1a667b14fe494c8ee4be0f47a340be" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.361553 885 rpcz_store.cc:276] 0505 07:54:06.312038 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.312108 (+ 70us) service_pool.cc:224] Handling call
0505 07:54:06.392216 (+ 80108us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0505 07:54:06.392248 (+ 32us) ts_tablet_manager.cc:682] Creating new metadata...
0505 07:54:07.014808 (+622560us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.361224 (+346416us) tablet_replica.cc:206] Creating consensus instance
0505 07:54:07.361402 (+ 178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:07.383430 907 raft_consensus.cc:2468] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.383618 909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d622612bdc404727a27b83769bf39738" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.383749 909 raft_consensus.cc:3060] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.383912 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "934cea6666b1420eb12fe4510d2466fe" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.384037 907 raft_consensus.cc:3060] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.384130 908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21480c42b73e4bf0ab2453b085a867c0" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.384291 908 raft_consensus.cc:2468] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.384564 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42360f8ee4ff47c2af8d287f792207de" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.385156 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3fc8a9d9f04636b1f31bb88411c1da" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.385308 916 raft_consensus.cc:2468] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.385349 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 0baf83b155494fddb3c25144608706f7 (table default.Table_5edc57ff557343629eb69ed0fe818ded [id=a92ad827e74542d98f9825d5aa198670]) successfully deleted
I20260505 07:54:07.388204 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46274a44a06b4306b5dac43c66c85a80" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.388355 917 raft_consensus.cc:3060] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.373536 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (table default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c [id=8a910f6a553a47ea87ec170e376982d1]). Expected version 2 got 0
I20260505 07:54:07.395736 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 83e62b7af9b847b7a9005e4345905c0d (table default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]). Expected version 3 got 0
I20260505 07:54:07.395920 908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f3544e74d941d286a2a0317b941ab6" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.396077 908 raft_consensus.cc:3060] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.396855 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]). Expected version 2 got 0
I20260505 07:54:07.398026 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:07.402268 899 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53998 (request call id 10) took 1087 ms. Trace:
I20260505 07:54:07.402359 899 rpcz_store.cc:276] 0505 07:54:06.315006 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.315072 (+ 66us) service_pool.cc:224] Handling call
0505 07:54:06.392370 (+ 77298us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0505 07:54:06.392400 (+ 30us) ts_tablet_manager.cc:682] Creating new metadata...
0505 07:54:07.040199 (+647799us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.402056 (+361857us) tablet_replica.cc:206] Creating consensus instance
0505 07:54:07.402220 (+ 164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:07.404856 2268 ts_tablet_manager.cc:1929] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:07.405019 2268 log.cc:1199] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/6da10815e0dc41a1a07b6786882fffd3
I20260505 07:54:07.405045 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:07.405344 2276 ts_tablet_manager.cc:1929] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:07.405417 2268 ts_tablet_manager.cc:1950] T 6da10815e0dc41a1a07b6786882fffd3 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:07.405965 2268 tablet_replica.cc:333] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:07.406175 2268 raft_consensus.cc:2243] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:07.406688 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:07.407702 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]). Expected version 2 got 0
I20260505 07:54:07.408190 894 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53998 (request call id 9) took 1094 ms. Trace:
I20260505 07:54:07.408280 894 rpcz_store.cc:276] 0505 07:54:06.314164 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.314228 (+ 64us) service_pool.cc:224] Handling call
0505 07:54:06.392294 (+ 78066us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0505 07:54:06.392325 (+ 31us) ts_tablet_manager.cc:682] Creating new metadata...
0505 07:54:07.019038 (+626713us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.408018 (+388980us) tablet_replica.cc:206] Creating consensus instance
0505 07:54:07.408171 (+ 153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:07.409196 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:07.409852 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5683d7366de84653ace56f0edcd3818c (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:07.410454 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]). Expected version 1 got 0
I20260505 07:54:07.411051 2302 raft_consensus.cc:493] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.411178 2302 raft_consensus.cc:515] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.411898 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:07.411955 2302 leader_election.cc:290] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.412177 2268 raft_consensus.cc:2272] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:07.413094 2268 ts_tablet_manager.cc:1916] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:07.413533 1357 catalog_manager.cc:5671] T 4af9def97a454f3f9b63d637db42c110 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:07.415179 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_268b9129e7fb4560bcbbb7e5427975fa [id=2c06ff8eed31432e87f08e9b59049652]). Expected version 1 got 0
I20260505 07:54:07.416368 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:07.417050 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_57b735b31fec4ba88439afddef823171 [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 1 got 0
I20260505 07:54:07.420065 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 6f25c3e1f7e442cba8c837b568c7b53f (DEFAULT_TABLE table=default.Table_89c343e38c814e91b9aa1da388b52770 [id=91983daad68744d2b41d11f378670fca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.420420 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f25c3e1f7e442cba8c837b568c7b53f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:07.420527 1348 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet de1a667b14fe494c8ee4be0f47a340be consensus state has no leader
I20260505 07:54:07.421017 1348 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_201f8409684e49be8bc5cd5249606250" } modify_external_catalogs: true
I20260505 07:54:07.421164 1348 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_201f8409684e49be8bc5cd5249606250" } modify_external_catalogs: true
W20260505 07:54:07.421918 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 62bf3a6bc0ac42a99a4181f820e3ca87 consensus state has no leader
I20260505 07:54:07.422246 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_af9bc6281a614df8908865ea0d5a7db9" } new_table_name: "default.Table_006d19a5a2b34032acd01663bdcc34e8" modify_external_catalogs: true
I20260505 07:54:07.426323 886 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53998 (request call id 25) took 1055 ms. Trace:
I20260505 07:54:07.426477 886 rpcz_store.cc:276] 0505 07:54:06.370599 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.866663 (+496064us) service_pool.cc:224] Handling call
0505 07:54:06.867068 (+ 405us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0505 07:54:06.867089 (+ 21us) ts_tablet_manager.cc:682] Creating new metadata...
0505 07:54:07.072119 (+205030us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.426150 (+354031us) tablet_replica.cc:206] Creating consensus instance
0505 07:54:07.426304 (+ 154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:07.427135 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (DEFAULT_TABLE table=default.Table_4ec0d26b7f5e4c25ac10858c5718448a [id=77cced330f69467cb7e54e309ff5a0db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.427500 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.428490 884 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53998 (request call id 12) took 1103 ms. Trace:
I20260505 07:54:07.428579 884 rpcz_store.cc:276] 0505 07:54:06.325477 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.325551 (+ 74us) service_pool.cc:224] Handling call
0505 07:54:06.392507 (+ 66956us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0505 07:54:06.392553 (+ 46us) ts_tablet_manager.cc:682] Creating new metadata...
0505 07:54:07.011624 (+619071us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.428340 (+416716us) tablet_replica.cc:206] Creating consensus instance
0505 07:54:07.428472 (+ 132us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260505 07:54:07.429435 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0c4342c2c6d142ad90b0270608007eda has no consensus state
I20260505 07:54:07.429737 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_67a8ca1fae934339bdfe388762b11ab2" } new_table_name: "default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd" modify_external_catalogs: true
I20260505 07:54:07.430548 892 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53998 (request call id 21) took 1074 ms. Trace:
I20260505 07:54:07.430630 892 rpcz_store.cc:276] 0505 07:54:06.355797 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.355855 (+ 58us) service_pool.cc:224] Handling call
0505 07:54:06.393253 (+ 37398us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0505 07:54:06.393284 (+ 31us) ts_tablet_manager.cc:682] Creating new metadata...
0505 07:54:07.017622 (+624338us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.430401 (+412779us) tablet_replica.cc:206] Creating consensus instance
0505 07:54:07.430529 (+ 128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:07.432302 2276 log.cc:1199] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/8eb7eaff8837485a8ae3ccb44c60471d
I20260505 07:54:07.432641 2276 ts_tablet_manager.cc:1950] T 8eb7eaff8837485a8ae3ccb44c60471d P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:07.432955 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 31) took 1026 ms. Trace:
I20260505 07:54:07.434518 1028 tablet_service.cc:1511] Processing CreateTablet for tablet bf0ce3e6c00541569d15c1b6ef347579 (DEFAULT_TABLE table=default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.434849 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf0ce3e6c00541569d15c1b6ef347579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.436754 1354 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 4662d4a3d88d4c2883a35048d6e11eb9
I20260505 07:54:07.437462 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_106ccea898e84018bd7f4c26b5960922"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.437952 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_106ccea898e84018bd7f4c26b5960922 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.440119 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 4662d4a3d88d4c2883a35048d6e11eb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:07.441522 2310 raft_consensus.cc:493] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.441684 2310 raft_consensus.cc:515] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.442507 2310 leader_election.cc:290] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.444365 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_c55e50477e284efca5d22c2b75683904"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:07.444664 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 4466d0ed977e4803b3b484155095c271 (DEFAULT_TABLE table=default.Table_1f316191e5124266935010a5cdcc7a18 [id=88b6ba8aaba145c2b3cac834bb9a988e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:07.444841 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c55e50477e284efca5d22c2b75683904 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.444957 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4466d0ed977e4803b3b484155095c271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.433032 2276 rpcz_store.cc:276] 0505 07:54:06.406537 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.936294 (+529757us) service_pool.cc:224] Handling call
0505 07:54:07.106006 (+169712us) ts_tablet_manager.cc:1151] Deleting tablet 8eb7eaff8837485a8ae3ccb44c60471d
0505 07:54:07.106016 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:07.186111 (+ 80095us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.405296 (+219185us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.432938 (+ 27642us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":169539,"thread_start_us":173117,"threads_started":1}
I20260505 07:54:07.448172 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 30) took 1050 ms. Trace:
I20260505 07:54:07.449172 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 8eb7eaff8837485a8ae3ccb44c60471d (table default.Table_66f28ba637f24448816a64c7d316edee [id=266ff796c7a74c23816dd9c422e9d005]) successfully deleted
I20260505 07:54:07.449716 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 4662d4a3d88d4c2883a35048d6e11eb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.448256 2276 rpcz_store.cc:276] 0505 07:54:06.397477 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.934539 (+537062us) service_pool.cc:224] Handling call
0505 07:54:07.448032 (+513493us) ts_tablet_manager.cc:1151] Deleting tablet abaa596871bb4a82aefc317932e91d35
0505 07:54:07.448046 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:07.448151 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":431868}
W20260505 07:54:07.450591 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet abaa596871bb4a82aefc317932e91d35 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abaa596871bb4a82aefc317932e91d35 already in progress: creating tablet
I20260505 07:54:07.451926 1018 tablet_service.cc:1511] Processing CreateTablet for tablet c01ebfd9a3514db89708ecc483be53eb (DEFAULT_TABLE table=default.Table_11048b568f9948f08dee95b3421474cb [id=b72a074c426c4c339daa2d28bab69672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.452224 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c01ebfd9a3514db89708ecc483be53eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.452145 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_8c181aa800d6451dbbc8db3e409c90ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.452544 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8c181aa800d6451dbbc8db3e409c90ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.452997 2276 tablet_replica.cc:333] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:07.453210 2276 raft_consensus.cc:2243] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:07.453369 2276 raft_consensus.cc:2272] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:07.453945 2276 ts_tablet_manager.cc:1916] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:07.454735 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet d896e290167b4d0081e8e8b831c8ae2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d896e290167b4d0081e8e8b831c8ae2a already in progress: creating tablet
W20260505 07:54:07.454983 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a9b7c6e9463342a5a73af12da4447f89 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9b7c6e9463342a5a73af12da4447f89 already in progress: creating tablet
W20260505 07:54:07.455161 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet ae829daad64545e687a9528e5c38b9ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae829daad64545e687a9528e5c38b9ed already in progress: creating tablet
W20260505 07:54:07.455380 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet b12d2d2b866348d781209fa787da7d7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b12d2d2b866348d781209fa787da7d7e already in progress: creating tablet
I20260505 07:54:07.458926 1018 tablet_service.cc:1511] Processing CreateTablet for tablet 9f541eff399641dd81a0771186805493 (DEFAULT_TABLE table=default.Table_a86abb495238445381ac0202804b4ca0 [id=4d3b183ab74a4078beb7583c9e60e2c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.459256 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f541eff399641dd81a0771186805493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.463574 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_f3446fea50fe4259aba56af3f6723b9c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.464177 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f3446fea50fe4259aba56af3f6723b9c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.464956 1032 tablet_service.cc:1511] Processing CreateTablet for tablet cf77f23a2ec647faa715fb912db3a995 (DEFAULT_TABLE table=default.Table_6b6489b552be4fef93c6cace622b4253 [id=04b3940c5da14dfa91a192fb78b3558d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.465246 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf77f23a2ec647faa715fb912db3a995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.470942 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_865701615b5f4c06884f469ea75d36f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.472000 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_865701615b5f4c06884f469ea75d36f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.472860 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet 4662d4a3d88d4c2883a35048d6e11eb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.473502 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f25c3e1f7e442cba8c837b568c7b53f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.473882 2268 ts_tablet_manager.cc:1929] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:07.473963 2268 log.cc:1199] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/d8c9eaaef250461ca7a37d0cf42b07ee
I20260505 07:54:07.474066 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.474316 2268 ts_tablet_manager.cc:1950] T d8c9eaaef250461ca7a37d0cf42b07ee P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:07.474552 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c8c6d524f3f44b2a0cc33d46a39d9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.474957 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf0ce3e6c00541569d15c1b6ef347579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.475102 2268 tablet_replica.cc:333] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:07.475284 2268 raft_consensus.cc:2243] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:07.475360 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4466d0ed977e4803b3b484155095c271. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.475445 2268 raft_consensus.cc:2272] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:07.475739 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fece4034c3a42169fae0c9f41dce0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.476130 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c01ebfd9a3514db89708ecc483be53eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.476325 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 6da10815e0dc41a1a07b6786882fffd3 (table default.Table_344c494090d44fa8b4102b525e7ca099 [id=d8050f3674fe49f7a8af288e6ea57a69]) successfully deleted
I20260505 07:54:07.476539 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f541eff399641dd81a0771186805493. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.476970 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2977a0a646448249d9fbe46f970d63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.477416 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf77f23a2ec647faa715fb912db3a995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.479506 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet d8c9eaaef250461ca7a37d0cf42b07ee (table default.Table_92db3a6ebf824942966e677a2cd23ce1 [id=4890bd76a6f04392848038377244273d]) successfully deleted
I20260505 07:54:07.479504 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_7ed74bd3c32f49ef8fbdeb8b065b2913"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.480314 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ed74bd3c32f49ef8fbdeb8b065b2913 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.481352 1355 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 54e4f7f895d943639653c8c32724baf2
I20260505 07:54:07.481947 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_da0ef200a5e645e0bef998207f1edcc7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.482327 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da0ef200a5e645e0bef998207f1edcc7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.489724 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 4c8c6d524f3f44b2a0cc33d46a39d9ea (DEFAULT_TABLE table=default.Table_7edd42954bfd448fbb723812869def42 [id=1f25e0a4dfd949659a907bc290c04772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.490211 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c8c6d524f3f44b2a0cc33d46a39d9ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.490911 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_e137326bc9684662bffdc5c59e122c6a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.491305 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e137326bc9684662bffdc5c59e122c6a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.496048 1018 tablet_service.cc:1511] Processing CreateTablet for tablet f2977a0a646448249d9fbe46f970d63b (DEFAULT_TABLE table=default.Table_ca2d7147756c4590a83c26ad0af48294 [id=c35d3d29588b42fc98bab9c8ebc3966b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.496405 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2977a0a646448249d9fbe46f970d63b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.497687 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_aa70b0997b464af5920413788a8cc6de"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.498095 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa70b0997b464af5920413788a8cc6de in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.500768 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 3fece4034c3a42169fae0c9f41dce0bb (DEFAULT_TABLE table=default.Table_46550be844b4470e8239e2ba41399a68 [id=b3ed9295a3634af8a75a142cd4a30363]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:07.501109 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fece4034c3a42169fae0c9f41dce0bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:07.505972 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_90d28c121c4c41568b9a2eb7f98e2294"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.506604 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_90d28c121c4c41568b9a2eb7f98e2294 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.511809 991 leader_election.cc:304] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.511917 2284 raft_consensus.cc:493] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.512077 2300 raft_consensus.cc:2804] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.512048 2284 raft_consensus.cc:515] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.512143 2300 raft_consensus.cc:493] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.512195 2300 raft_consensus.cc:3060] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.512533 2284 leader_election.cc:290] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.514369 2300 raft_consensus.cc:515] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.514732 1358 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_4ab0ac01b8bf49ef96683dca41dfadd4 [id=6cb8bc8043474bad9f4278b7eef7c71a] alter complete (version 3)
I20260505 07:54:07.515162 2300 leader_election.cc:290] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:07.515125 854 leader_election.cc:343] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.515455 854 leader_election.cc:304] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.515834 1866 raft_consensus.cc:2749] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:07.516268 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 54e4f7f895d943639653c8c32724baf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:07.516688 2268 ts_tablet_manager.cc:1916] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:07.517053 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a73dbb979344805876efbafbf1b2985" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.521406 991 leader_election.cc:304] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.521803 991 leader_election.cc:304] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.522094 2308 raft_consensus.cc:2804] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.522179 2308 raft_consensus.cc:493] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.522282 2308 raft_consensus.cc:3060] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:07.524669 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5006965919824adbbb23d744fb9ff335 consensus state has no leader
W20260505 07:54:07.525892 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 9cbfb2345a1a4b218c2dddbae6667e47 consensus state has no leader
I20260505 07:54:07.526345 2308 raft_consensus.cc:515] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:07.526654 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet f842d8ac6b2142ffa69945c6087be34b consensus state has no leader
I20260505 07:54:07.527060 2308 leader_election.cc:290] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.527251 2308 raft_consensus.cc:2804] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.527361 2308 raft_consensus.cc:493] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.527477 2308 raft_consensus.cc:3060] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.527594 2089 consensus_queue.cc:1048] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:07.528443 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 01431ad112e8445487823efe8feb0dbe consensus state has no leader
I20260505 07:54:07.528942 2089 consensus_queue.cc:1048] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:07.529645 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet e278b86e7b5d444ebad8d0f6921d4c62
I20260505 07:54:07.529881 2201 consensus_queue.cc:1048] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:07.529651 1352 catalog_manager.cc:5953] Not sending DeleteTablet requests; no consensus state for tablet a32eeaca77c1475f879465cb1046543b
I20260505 07:54:07.531450 1176 tablet_bootstrap.cc:492] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
W20260505 07:54:07.531819 1353 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 68e608ac01ef4a88906e585bd7ee25ac consensus state has no leader
I20260505 07:54:07.534574 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feeafd614b3e4281a8e45b950e699b63" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.535943 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727cacd242844455915bd9963d2db19f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.536697 747 tablet_service.cc:1558] Processing DeleteTablet for tablet e278b86e7b5d444ebad8d0f6921d4c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:07.537760 2137 ts_tablet_manager.cc:1929] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:07.537863 2137 log.cc:1199] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/8eb7eaff8837485a8ae3ccb44c60471d
I20260505 07:54:07.538190 2137 ts_tablet_manager.cc:1950] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:07.538666 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 8eb7eaff8837485a8ae3ccb44c60471d (table default.Table_66f28ba637f24448816a64c7d316edee [id=266ff796c7a74c23816dd9c422e9d005]) successfully deleted
W20260505 07:54:07.539265 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet d896e290167b4d0081e8e8b831c8ae2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d896e290167b4d0081e8e8b831c8ae2a already in progress: creating tablet
I20260505 07:54:07.539532 2137 tablet_replica.cc:333] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
W20260505 07:54:07.539644 776 consensus_meta.cc:287] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent flushing consensus metadata: real 0.790s user 0.018s sys 0.003s
I20260505 07:54:07.539732 2137 raft_consensus.cc:2243] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:07.539728 2308 raft_consensus.cc:515] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.539880 2137 raft_consensus.cc:2272] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:07.540370 2308 leader_election.cc:290] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.540457 2137 ts_tablet_manager.cc:1916] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:07.540421 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe19306f7df49fdb771d21f6560e697" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.544104 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4e5af08ddc4c77b5640b8295b236b1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.544276 908 raft_consensus.cc:2468] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.545296 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29ba55a846548679cb3b86305d0d6ed" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.546500 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "297537a92684425d8508f7b3b4c9d379" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.546844 918 raft_consensus.cc:2468] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.548331 910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29ba55a846548679cb3b86305d0d6ed" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.548487 910 raft_consensus.cc:2468] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.548761 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f121fdd001ec4ea38ccd24a6afd9e51c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.551055 917 raft_consensus.cc:2468] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.539844 776 raft_consensus.cc:2468] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.552394 907 raft_consensus.cc:2468] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.553638 2201 consensus_queue.cc:1048] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:07.565142 991 leader_election.cc:304] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.565558 991 leader_election.cc:304] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.566224 991 leader_election.cc:304] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.566468 2310 raft_consensus.cc:2804] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.566548 2310 raft_consensus.cc:493] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.566617 2310 raft_consensus.cc:3060] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.569083 2310 raft_consensus.cc:515] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.569825 2310 leader_election.cc:290] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.569991 2310 raft_consensus.cc:2749] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:07.570367 2310 raft_consensus.cc:2804] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.570663 2310 raft_consensus.cc:697] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.570924 2310 consensus_queue.cc:237] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.572578 884 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.572754 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 09b46ff0101a4e529077b9e30492ace7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.575068 1176 ts_tablet_manager.cc:1403] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.579s user 0.018s sys 0.013s
I20260505 07:54:07.575825 1176 raft_consensus.cc:359] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.575979 1176 raft_consensus.cc:385] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:07.576041 1176 raft_consensus.cc:740] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:07.575897 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:07.576210 1176 consensus_queue.cc:260] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.582476 909 raft_consensus.cc:2468] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.584218 886 tablet_service.cc:1558] Processing DeleteTablet for tablet 02e06d8815034d91986cef12964860ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.584678 900 tablet_service.cc:1558] Processing DeleteTablet for tablet f200abef13ac42698a9b96fc8af82ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.585744 884 tablet_service.cc:1558] Processing DeleteTablet for tablet e7887c3462f74740b314e3bec36e280d with delete_type TABLET_DATA_DELETED (replaced by tablet fba1d19f902a4a0a8b581135f6611129) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.586107 892 tablet_service.cc:1558] Processing DeleteTablet for tablet a9494c3b6a99487596a030856b371b68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6fe19306f7df49fdb771d21f6560e697) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.585179 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 54e4f7f895d943639653c8c32724baf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.593320 884 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.593508 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.594780 895 tablet_service.cc:1558] Processing DeleteTablet for tablet a9b7c6e9463342a5a73af12da4447f89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.596347 991 leader_election.cc:304] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.596691 991 leader_election.cc:304] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.606056 2284 raft_consensus.cc:493] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.606189 991 leader_election.cc:304] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.606186 2284 raft_consensus.cc:515] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.597059 2300 raft_consensus.cc:2804] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.606353 2300 raft_consensus.cc:493] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.606419 2300 raft_consensus.cc:3060] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.606784 2284 leader_election.cc:290] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.597154 2308 raft_consensus.cc:493] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:07.607113 856 leader_election.cc:343] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.607141 2308 raft_consensus.cc:515] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.607834 2308 leader_election.cc:290] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.610246 2300 raft_consensus.cc:515] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.611083 2300 leader_election.cc:290] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.598383 1181 tablet_bootstrap.cc:492] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:07.613099 1181 ts_tablet_manager.cc:1403] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.321s user 0.004s sys 0.010s
I20260505 07:54:07.616946 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b4e886bebc4ab6a3c95290d18b4229" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.617846 889 tablet_service.cc:1558] Processing DeleteTablet for tablet e82e9ae417784b7e83f4843b129d0580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:07.618475 856 leader_election.cc:343] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.622366 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a403e4708e4725899c1cd5b9fb0e8b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.623008 877 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:33748 (request call id 114) took 1266 ms. Trace:
I20260505 07:54:07.623935 1176 ts_tablet_manager.cc:1434] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.049s user 0.002s sys 0.000s
I20260505 07:54:07.624203 1176 tablet_bootstrap.cc:492] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:07.624521 889 tablet_service.cc:1558] Processing DeleteTablet for tablet 26f3544e74d941d286a2a0317b941ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.624626 877 rpcz_store.cc:276] 0505 07:54:06.356740 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.435794 (+ 79054us) service_pool.cc:224] Handling call
0505 07:54:07.057743 (+621949us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x6160000d4554. stack: 00007f1644039086 00007f1644038e40 00007f1642ff23ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007f164ff4b8c0 00007f1653e06eaf 00007f164e21ec36 00007f164e21e965 00007f164b02c10e 00007f164b02afee 00007f164b031228 00007f164b031b8f 00007f164b0319b8 0000000000aa4cac
0505 07:54:07.363457 (+305714us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x616000230e54. stack: 00007f1644039086 00007f1644038e40 00007f1642ff23ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007f164ff4b8c0 00007f1653e06eaf 00007f164e21ec36 00007f164e21e965 00007f164b02c10e 00007f164b02afee 00007f164b031228 00007f164b031b8f 00007f164b0319b8 0000000000aa4cac
0505 07:54:07.622969 (+259512us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211508736}
I20260505 07:54:07.629110 891 tablet_service.cc:1558] Processing DeleteTablet for tablet e278b86e7b5d444ebad8d0f6921d4c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:07.630111 1172 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.505s user 0.016s sys 0.062s
W20260505 07:54:07.630190 1172 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.505s user 0.016s sys 0.062s
I20260505 07:54:07.642376 1172 tablet_bootstrap.cc:492] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:07.642606 2307 raft_consensus.cc:2804] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.642642 1172 ts_tablet_manager.cc:1403] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.531s user 0.016s sys 0.065s
I20260505 07:54:07.642719 2307 raft_consensus.cc:493] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.642796 2307 raft_consensus.cc:3060] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.643438 1352 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_db54454473d84a83b6c77d06a1f09618" } new_table_name: "default.Table_925028e721e24ba68f31fe1040d4c2fd" modify_external_catalogs: true
I20260505 07:54:07.643885 1350 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_043f3d0ae81c46a9a77b3264088fd1a9" } new_table_name: "default.Table_947d0109c8c343418574a18377c6cccd" modify_external_catalogs: true
I20260505 07:54:07.644312 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_95fae21889c84b61ad67610d209031b2" } new_table_name: "default.Table_79956c61f6444a4fad1ca4b7950feb9d" modify_external_catalogs: true
I20260505 07:54:07.644516 1355 master_service.cc:954] ReplaceTablet: received request to replace tablet 25072055f5534078adf5b5025614f951 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:07.645685 2307 raft_consensus.cc:515] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.646155 2302 raft_consensus.cc:2804] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.646315 2302 raft_consensus.cc:697] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.646564 2302 consensus_queue.cc:237] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:07.647778 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 08c6ed755fc64080a2310fe9a4e0a68b (table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet 08c6ed755fc64080a2310fe9a4e0a68b consensus state has no leader
I20260505 07:54:07.648128 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 06904a54b1c5463da31af0bc00fb606f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.649008 896 tablet_service.cc:1558] Processing DeleteTablet for tablet a7d8265c05b74b28959d0de41fad1fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.649221 882 tablet_service.cc:1558] Processing DeleteTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.649354 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 9044763bed1a41739ecce4def4cca1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.649663 888 tablet_service.cc:1558] Processing DeleteTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e08cf03ea0b495a94c6566ea6c8caad) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.649900 889 tablet_service.cc:1558] Processing DeleteTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf with delete_type TABLET_DATA_DELETED (replaced by tablet 3a73dbb979344805876efbafbf1b2985) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.649942 2307 leader_election.cc:290] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.650154 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 17327cee69a24e0cb493ff55f948bf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.650424 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 840eebf9e7404bf7807b7e51996957e5 with delete_type TABLET_DATA_DELETED (replaced by tablet bf0ce3e6c00541569d15c1b6ef347579) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.650161 2276 ts_tablet_manager.cc:1929] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:07.650835 2276 log.cc:1199] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/85d485ae2d1d4d76a6a5d988cec8291a
I20260505 07:54:07.651427 2276 ts_tablet_manager.cc:1950] T 85d485ae2d1d4d76a6a5d988cec8291a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
W20260505 07:54:07.653362 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e6a5f823db6143319d743d363a77ea47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6a5f823db6143319d743d363a77ea47 already in progress: creating tablet
I20260505 07:54:07.653733 2276 tablet_replica.cc:333] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:07.654088 2276 raft_consensus.cc:2243] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:07.654327 1181 raft_consensus.cc:359] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.654420 2276 raft_consensus.cc:2272] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:07.655154 1181 raft_consensus.cc:385] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:07.655279 2276 ts_tablet_manager.cc:1916] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:07.655510 1181 raft_consensus.cc:740] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:07.656651 1181 consensus_queue.cc:260] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.657196 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a06f77255544d7dabb71a4f70ada809 with delete_type TABLET_DATA_DELETED (replaced by tablet 12841efcef6041d89fc6d7d9b90448c0) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.657629 1181 ts_tablet_manager.cc:1434] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.044s user 0.002s sys 0.000s
W20260505 07:54:07.657945 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_95fae21889c84b61ad67610d209031b2 [id=b131fb4265684cd2b6506a7abeb29871], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 76939b7978ee4f7ea0ad29312de2a31a consensus state has no leader
W20260505 07:54:07.657967 911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s user 0.015s sys 0.008s
W20260505 07:54:07.658548 911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s user 0.015s sys 0.008s
I20260505 07:54:07.658627 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db162dfad2448e8ac428b3d3fbe17f7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.658936 901 tablet_service.cc:1558] Processing DeleteTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:07.658228 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 52054.7us
I20260505 07:54:07.659289 1181 tablet_bootstrap.cc:492] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:07.661121 891 tablet_service.cc:1558] Processing DeleteTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.661279 889 tablet_service.cc:1558] Processing DeleteTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 with delete_type TABLET_DATA_DELETED (replaced by tablet d5083f98e8e14ccf99a19cdf2bc59018) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.661448 901 tablet_service.cc:1558] Processing DeleteTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e with delete_type TABLET_DATA_DELETED (replaced by tablet dd3fc8a9d9f04636b1f31bb88411c1da) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.662068 1181 tablet_bootstrap.cc:654] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:07.663755 1172 raft_consensus.cc:359] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.663978 1172 raft_consensus.cc:385] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:07.664088 1172 raft_consensus.cc:740] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:07.666442 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 78a6470720984761a8606abeafbef992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.666601 889 tablet_service.cc:1558] Processing DeleteTablet for tablet 003cd4472b424aacbba8c5f89900e228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.659668 991 leader_election.cc:304] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.670946 2268 ts_tablet_manager.cc:1929] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:07.671257 991 leader_election.cc:304] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.671392 2300 raft_consensus.cc:2804] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
W20260505 07:54:07.676293 988 leader_election.cc:343] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.676440 988 leader_election.cc:304] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.664983 1172 consensus_queue.cc:260] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.677136 2308 raft_consensus.cc:2749] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260505 07:54:07.677757 1172 ts_tablet_manager.cc:1434] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.035s user 0.001s sys 0.001s
I20260505 07:54:07.681627 1172 tablet_bootstrap.cc:492] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:07.682863 2300 raft_consensus.cc:697] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.683699 889 tablet_service.cc:1558] Processing DeleteTablet for tablet 8eb7eaff8837485a8ae3ccb44c60471d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:07.685685 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet f840d5f58161468385d5c4f116b78c3d consensus state has no leader
I20260505 07:54:07.688777 889 tablet_service.cc:1558] Processing DeleteTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.690026 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:07.690804 2089 consensus_queue.cc:1048] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:07.693060 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 0e08cf03ea0b495a94c6566ea6c8caad consensus state has no leader
W20260505 07:54:07.694772 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 09019fdf60374b11bf3427f21a55b5e0 consensus state has no leader
I20260505 07:54:07.696902 889 tablet_service.cc:1558] Processing DeleteTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.700495 2302 raft_consensus.cc:2804] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.701639 889 tablet_service.cc:1558] Processing DeleteTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.703965 1172 tablet_bootstrap.cc:654] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:07.708101 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 17b4e886bebc4ab6a3c95290d18b4229 consensus state has no leader
I20260505 07:54:07.712901 2302 raft_consensus.cc:697] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.713110 889 tablet_service.cc:1558] Processing DeleteTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.713178 2302 consensus_queue.cc:237] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.713275 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 12501fdf2cc5491cb1a06db490e70810 with delete_type TABLET_DATA_DELETED (replaced by tablet d622612bdc404727a27b83769bf39738) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.713418 891 tablet_service.cc:1558] Processing DeleteTablet for tablet f1058fcb9fa6457a85489049ca1d9ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.713562 888 tablet_service.cc:1558] Processing DeleteTablet for tablet 141665fcd59048bf926acd0287f1853e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.713696 895 tablet_service.cc:1558] Processing DeleteTablet for tablet d896e290167b4d0081e8e8b831c8ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.717413 2300 consensus_queue.cc:237] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.717577 1172 tablet_bootstrap.cc:492] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:07.718256 1172 ts_tablet_manager.cc:1403] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.037s user 0.000s sys 0.004s
I20260505 07:54:07.719528 1172 raft_consensus.cc:359] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.720394 1172 raft_consensus.cc:385] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:07.720602 1172 raft_consensus.cc:740] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:07.720942 1172 consensus_queue.cc:260] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.721969 1172 ts_tablet_manager.cc:1434] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260505 07:54:07.723891 917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "036d306d592f41cb96814aa60e30551e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.724035 917 raft_consensus.cc:3060] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.724087 910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21480c42b73e4bf0ab2453b085a867c0" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.724226 910 raft_consensus.cc:3060] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.725068 908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7001482d90364dbdb8b9c5bfb98e1cd4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.725217 908 raft_consensus.cc:2468] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.725477 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "297537a92684425d8508f7b3b4c9d379" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.725616 919 raft_consensus.cc:3060] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.726881 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet e278b86e7b5d444ebad8d0f6921d4c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.727116 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de1a667b14fe494c8ee4be0f47a340be" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.727236 912 raft_consensus.cc:3060] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.728348 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29ba55a846548679cb3b86305d0d6ed" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.728474 913 raft_consensus.cc:3060] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.728830 917 raft_consensus.cc:2468] T 036d306d592f41cb96814aa60e30551e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.729664 910 raft_consensus.cc:2468] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.731846 1018 tablet_service.cc:1558] Processing DeleteTablet for tablet 54e4f7f895d943639653c8c32724baf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:07.732777 912 raft_consensus.cc:2468] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.735277 919 raft_consensus.cc:2468] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.735397 913 raft_consensus.cc:2468] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.744321 2308 consensus_queue.cc:1048] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:07.745970 2276 ts_tablet_manager.cc:1929] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:07.746304 2276 log.cc:1199] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/accb2dedc30e494ebd22c33594cee661
I20260505 07:54:07.746665 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3fc8a9d9f04636b1f31bb88411c1da" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.746855 918 raft_consensus.cc:3060] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.749658 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feeafd614b3e4281a8e45b950e699b63" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.750049 918 raft_consensus.cc:2468] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.761691 1176 tablet_bootstrap.cc:654] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:07.762197 717 consensus_peers.cc:597] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8c9eaaef250461ca7a37d0cf42b07ee. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:07.762878 991 leader_election.cc:304] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.763185 2308 raft_consensus.cc:2804] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.763316 2308 raft_consensus.cc:493] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.763446 2308 raft_consensus.cc:3060] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:07.766564 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 72da22d790624b07bfca6f6b6301c925 (table default.Table_201f8409684e49be8bc5cd5249606250 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 72da22d790624b07bfca6f6b6301c925 consensus state has no leader
W20260505 07:54:07.770952 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 1001dd888ecd4166ba217506739f71e8 consensus state has no leader
W20260505 07:54:07.773687 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 6f207a3ffbc94b01abd9e9d6837f5c81 consensus state has no leader
W20260505 07:54:07.774134 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet dd3fc8a9d9f04636b1f31bb88411c1da consensus state has no leader
W20260505 07:54:07.774829 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 4b9963a0f1a54ee9a73486bb1801ee74 consensus state has no leader
W20260505 07:54:07.776022 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 310ef5aef7e44ec5a062a01d2a175a11 consensus state has no leader
W20260505 07:54:07.776368 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 90a13ca612f241fc97ea12124677a0b1 consensus state has no leader
W20260505 07:54:07.777642 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52], current schema version=4): Not found: failed to reset TS proxy: No leader found: Tablet 423c22891bed462d89e9ec51a079f0f5 consensus state has no leader
W20260505 07:54:07.777817 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 54e4f7f895d943639653c8c32724baf2 (table default.Table_ceaf4d54d57542928716e3056eac2bde [id=54621fbfaf6a4b4a874207bd986b77be], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet 54e4f7f895d943639653c8c32724baf2 consensus state has no leader
W20260505 07:54:07.777976 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet f106a6d4855148fe83b1e90ea9f27d39 consensus state has no leader
W20260505 07:54:07.778223 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 7235a3ccef904913bbd67c403aac3a87 consensus state has no leader
W20260505 07:54:07.778404 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet f84f464e83cb412eade9978ea99f6685 consensus state has no leader
W20260505 07:54:07.778548 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet 66082a7a75e345639e7a268727922fbb consensus state has no leader
W20260505 07:54:07.779089 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet c837877946604ef29f95dac9dd449bf3 consensus state has no leader
W20260505 07:54:07.779451 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet d5083f98e8e14ccf99a19cdf2bc59018 consensus state has no leader
I20260505 07:54:07.780063 2308 raft_consensus.cc:515] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:07.781129 1358 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.2:57579 (request call id 153) took 2045 ms (client timeout 1000 ms). Trace:
W20260505 07:54:07.781236 1358 rpcz_store.cc:269] 0505 07:54:05.735922 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:05.735977 (+ 55us) service_pool.cc:224] Handling call
0505 07:54:07.781076 (+2045099us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:06.127601 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:06.127745 (+ 144us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:06.127753 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:06.127755 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:06.128080 (+ 325us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:06.128096 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:06.128101 (+ 5us) tablet.cc:685] Acquiring locks for 11 operations
0505 07:54:06.128358 (+ 257us) tablet.cc:701] Row locks acquired
0505 07:54:06.128361 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:06.128408 (+ 47us) write_op.cc:270] Start()
0505 07:54:06.128473 (+ 65us) write_op.cc:276] Timestamp: P: 1777967646128383 usec, L: 0
0505 07:54:06.128490 (+ 17us) op_driver.cc:348] REPLICATION: starting
0505 07:54:06.128709 (+ 219us) log.cc:844] Serialized 4268 byte log entry
0505 07:54:06.132920 (+ 4211us) op_driver.cc:464] REPLICATION: finished
0505 07:54:06.133068 (+ 148us) write_op.cc:301] APPLY: starting
0505 07:54:06.133084 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:06.133218 (+ 134us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:06.133251 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:06.133411 (+ 160us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:06.133456 (+ 45us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0505 07:54:06.133464 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:06.133567 (+ 103us) log.cc:844] Serialized 83 byte log entry
0505 07:54:06.133650 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:06.133684 (+ 34us) write_op.cc:454] Released schema lock
0505 07:54:06.133697 (+ 13us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:06.133778 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:06.133783 (+ 5us) write_op.cc:454] Released schema lock
Related trace 'op':
0505 07:54:06.617739 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:06.617872 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:06.617879 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:06.617882 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:06.617990 (+ 108us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:06.618004 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:06.618009 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:06.618074 (+ 65us) tablet.cc:701] Row locks acquired
0505 07:54:06.618077 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:06.618112 (+ 35us) write_op.cc:270] Start()
0505 07:54:06.618145 (+ 33us) write_op.cc:276] Timestamp: P: 1777967646618093 usec, L: 0
0505 07:54:06.618149 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:06.618292 (+ 143us) log.cc:844] Serialized 356 byte log entry
0505 07:54:06.638814 (+ 20522us) op_driver.cc:464] REPLICATION: finished
0505 07:54:06.639565 (+ 751us) write_op.cc:301] APPLY: starting
0505 07:54:06.639574 (+ 9us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:06.639622 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:06.639675 (+ 53us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:06.639712 (+ 37us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:06.639737 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:06.639744 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:06.639795 (+ 51us) log.cc:844] Serialized 23 byte log entry
0505 07:54:06.640738 (+ 943us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:06.640751 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:06.640756 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:06.657745 (+ 16989us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:06.657753 (+ 8us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":188,"child_traces":[["op",{"apply.queue_time_us":98,"apply.run_cpu_time_us":701,"apply.run_wall_time_us":700,"mutex_wait_us":529,"num_ops":11,"prepare.queue_time_us":15501,"prepare.run_cpu_time_us":1291,"prepare.run_wall_time_us":1810,"raft.queue_time_us":624,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":297,"replication_time_us":4399,"spinlock_wait_cycles":1792}],["op",{"apply.queue_time_us":721,"apply.run_cpu_time_us":348,"apply.run_wall_time_us":1241,"num_ops":1,"prepare.queue_time_us":104878,"prepare.run_cpu_time_us":7860,"prepare.run_wall_time_us":40000,"raft.queue_time_us":1666,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"replication_time_us":20651,"thread_start_us":39315,"threads_started":1}]]}
I20260505 07:54:07.774156 1354 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c" } modify_external_catalogs: true
I20260505 07:54:07.784240 2137 ts_tablet_manager.cc:1929] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:07.784215 1354 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c" } modify_external_catalogs: true
I20260505 07:54:07.785328 2302 raft_consensus.cc:493] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.785562 2302 raft_consensus.cc:515] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.785728 2137 log.cc:1199] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/accb2dedc30e494ebd22c33594cee661
I20260505 07:54:07.786415 2302 leader_election.cc:290] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.788861 991 leader_election.cc:304] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.791239 1181 tablet_bootstrap.cc:492] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:07.791436 1181 ts_tablet_manager.cc:1403] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.132s user 0.003s sys 0.001s
I20260505 07:54:07.795832 2308 leader_election.cc:290] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:07.796304 1317 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet f84f464e83cb412eade9978ea99f6685 consensus state has no leader
I20260505 07:54:07.796641 2105 raft_consensus.cc:2804] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.796777 2105 raft_consensus.cc:697] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.797050 2105 consensus_queue.cc:237] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.768157 895 tablet_service.cc:1558] Processing DeleteTablet for tablet abaa596871bb4a82aefc317932e91d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.798771 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 4258df74ea0841449846cdf1f6e00b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.798944 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f207a3ffbc94b01abd9e9d6837f5c81) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.799108 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 02412333633a4cdebe95a144fa6c363c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.799232 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.799358 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 46274a44a06b4306b5dac43c66c85a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.799513 895 tablet_service.cc:1558] Processing DeleteTablet for tablet e29ba55a846548679cb3b86305d0d6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.799661 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 3075e87e078f4d2aa1d329d18902354b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.799790 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 27bacc97d7534284970f218cf68ac75b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.799925 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c4fc18f6faa488882d9bffb30793cdd) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.800062 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.800206 895 tablet_service.cc:1558] Processing DeleteTablet for tablet e010f23c4df647ab863e03c3cfed7898 with delete_type TABLET_DATA_DELETED (replaced by tablet 09019fdf60374b11bf3427f21a55b5e0) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.800341 895 tablet_service.cc:1558] Processing DeleteTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.800468 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ad8a79245ad47738d079752e51aed2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.800614 895 tablet_service.cc:1558] Processing DeleteTablet for tablet d622612bdc404727a27b83769bf39738 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.800778 895 tablet_service.cc:1558] Processing DeleteTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.800915 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 727cacd242844455915bd9963d2db19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.801056 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 01431ad112e8445487823efe8feb0dbe) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.801213 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 297537a92684425d8508f7b3b4c9d379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.801340 895 tablet_service.cc:1558] Processing DeleteTablet for tablet df1b62a30b2842b897a04c793bb5e7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.801472 895 tablet_service.cc:1558] Processing DeleteTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.801616 895 tablet_service.cc:1558] Processing DeleteTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.801730 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 080b2984fcf2478a8def6540da63554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.801865 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 472441bd29704be59aabb0f7a4964bb0 with delete_type TABLET_DATA_DELETED (replaced by tablet e7887c3462f74740b314e3bec36e280d) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.801987 895 tablet_service.cc:1558] Processing DeleteTablet for tablet aab3850799254dda9716fd5bf9404159 with delete_type TABLET_DATA_DELETED (replaced by tablet 5006965919824adbbb23d744fb9ff335) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.802109 895 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.802145 991 leader_election.cc:304] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.802208 895 tablet_service.cc:1558] Processing DeleteTablet for tablet abe5f6f0774545b296bf1b03a08e145d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.768308 888 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.772235 891 tablet_service.cc:1558] Processing DeleteTablet for tablet e8bf49b787ef4de095448751464708a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.772370 901 tablet_service.cc:1558] Processing DeleteTablet for tablet 323269e708964848a94c64e6a781a860 with delete_type TABLET_DATA_DELETED (replaced by tablet e010f23c4df647ab863e03c3cfed7898) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.772500 889 tablet_service.cc:1558] Processing DeleteTablet for tablet 60772a9697ba417b879dc6025eeb8eec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.772644 882 tablet_service.cc:1558] Processing DeleteTablet for tablet 0baf83b155494fddb3c25144608706f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.772760 896 tablet_service.cc:1558] Processing DeleteTablet for tablet feeafd614b3e4281a8e45b950e699b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.772862 898 tablet_service.cc:1558] Processing DeleteTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.772962 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 93d878ae0ddb42319560f74e0a490321 with delete_type TABLET_DATA_DELETED (replaced by tablet bc9ae6404dd64bd8bf4898768e94e4f1) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.775326 884 tablet_service.cc:1558] Processing DeleteTablet for tablet b5a0cbf9c3d74cea92920189da332c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.776226 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 397d33e908af4a0bb8dde26d273be01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.776664 900 tablet_service.cc:1558] Processing DeleteTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.777765 886 tablet_service.cc:1558] Processing DeleteTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.778082 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4bc3154704ba2bfde83bca10f812d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.778335 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b57c1c618484d5d86a846fb982a2240 with delete_type TABLET_DATA_DELETED (replaced by tablet de1a667b14fe494c8ee4be0f47a340be) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.778596 893 tablet_service.cc:1558] Processing DeleteTablet for tablet c1a61c1d03d544c8aa01537ead97f83c with delete_type TABLET_DATA_DELETED (replaced by tablet 080b2984fcf2478a8def6540da63554d) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.778846 885 tablet_service.cc:1558] Processing DeleteTablet for tablet d77ff90ec2ff4b6eb1795d149459c23a with delete_type TABLET_DATA_DELETED (replaced by tablet e5eaa91ecc1f43d18b313b6be4cc211e) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.802980 2307 raft_consensus.cc:2804] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.779101 897 tablet_service.cc:1558] Processing DeleteTablet for tablet bc9ae6404dd64bd8bf4898768e94e4f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.780586 890 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f1feb0387bc4afead72346398dbe9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.781196 883 tablet_service.cc:1558] Processing DeleteTablet for tablet d3d11bd19f074bbab101448b45426b7a with delete_type TABLET_DATA_DELETED (replaced by tablet 76939b7978ee4f7ea0ad29312de2a31a) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:07.803128 2307 raft_consensus.cc:697] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.803381 2307 consensus_queue.cc:237] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.806141 991 leader_election.cc:304] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.807046 2307 raft_consensus.cc:2804] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.807065 1909 raft_consensus.cc:493] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.807175 2307 raft_consensus.cc:697] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.807220 1909 raft_consensus.cc:515] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.807435 2307 consensus_queue.cc:237] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.807946 1909 leader_election.cc:290] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:07.808373 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba616702dc6040bb805bb3ffc8a8a871" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:07.808905 856 leader_election.cc:343] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.818354 1181 raft_consensus.cc:359] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.818661 1181 raft_consensus.cc:385] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260505 07:54:07.818733 2257 log.cc:927] Time spent T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Append to log took a long time: real 0.145s user 0.000s sys 0.000s
I20260505 07:54:07.818802 1181 raft_consensus.cc:740] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:07.819223 1181 consensus_queue.cc:260] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.789373 2293 raft_consensus.cc:493] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.819949 1181 ts_tablet_manager.cc:1434] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.028s user 0.002s sys 0.000s
I20260505 07:54:07.820009 2293 raft_consensus.cc:515] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.821012 2268 log.cc:1199] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/aab3850799254dda9716fd5bf9404159
I20260505 07:54:07.821548 2268 ts_tablet_manager.cc:1950] T aab3850799254dda9716fd5bf9404159 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:07.821997 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 4) took 1063 ms. Trace:
I20260505 07:54:07.822371 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5191f9a357de4607aab6c7a8b4bcccdb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.822708 920 raft_consensus.cc:3060] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.822243 2268 rpcz_store.cc:276] 0505 07:54:06.758353 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.758401 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:07.474664 (+716263us) ts_tablet_manager.cc:1151] Deleting tablet aab3850799254dda9716fd5bf9404159
0505 07:54:07.474670 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:07.620613 (+145943us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.670892 (+ 50279us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.821938 (+151046us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":716093}
I20260505 07:54:07.823267 2268 tablet_replica.cc:333] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:07.823524 2268 raft_consensus.cc:2243] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:07.824222 2268 raft_consensus.cc:2272] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:07.824970 2268 ts_tablet_manager.cc:1916] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:07.826844 920 raft_consensus.cc:2468] T 5191f9a357de4607aab6c7a8b4bcccdb P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:07.828887 991 leader_election.cc:304] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.829233 2295 raft_consensus.cc:2804] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.829381 2295 raft_consensus.cc:697] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.829653 2295 consensus_queue.cc:237] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.847440 991 leader_election.cc:304] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.847947 991 leader_election.cc:304] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.848464 1878 raft_consensus.cc:2804] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.848628 1878 raft_consensus.cc:697] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.848845 1878 consensus_queue.cc:237] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.857205 2250 raft_consensus.cc:2804] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:07.857345 2250 raft_consensus.cc:697] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:07.857654 2250 consensus_queue.cc:237] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.857982 1358 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_f56ca420b05b4508a41ee72179b9c93f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.858548 1358 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f56ca420b05b4508a41ee72179b9c93f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.863372 2089 raft_consensus.cc:493] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:07.868924 2293 leader_election.cc:290] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.869678 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "297537a92684425d8508f7b3b4c9d379" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.863528 2089 raft_consensus.cc:515] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:07.870797 2334 tablet.cc:1724] T 8d1ce00e487d4702a6affcad57a7fbe1 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:07.871665 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "934cea6666b1420eb12fe4510d2466fe" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:07.873962 2089 leader_election.cc:290] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.880754 1358 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_2634c105c2c041959e881249dddecdaf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:07.883673 1358 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2634c105c2c041959e881249dddecdaf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:07.885977 2325 tablet.cc:1724] T 8d1ce00e487d4702a6affcad57a7fbe1 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:07.891227 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36b33d265ba04e93993c7e0a4efea1a4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:07.891638 907 raft_consensus.cc:2468] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:07.893563 2137 ts_tablet_manager.cc:1950] T accb2dedc30e494ebd22c33594cee661 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:07.896942 2137 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 36) took 1281 ms. Trace:
I20260505 07:54:07.897025 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7001482d90364dbdb8b9c5bfb98e1cd4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.897246 916 raft_consensus.cc:3060] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.899971 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet aab3850799254dda9716fd5bf9404159 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]) successfully deleted
I20260505 07:54:07.901150 991 leader_election.cc:304] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.902822 2308 raft_consensus.cc:2804] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:07.902973 2308 raft_consensus.cc:493] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:07.903131 2308 raft_consensus.cc:3060] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.907338 2308 raft_consensus.cc:515] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.897094 2137 rpcz_store.cc:276] 0505 07:54:06.615076 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.615116 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:07.539489 (+924373us) ts_tablet_manager.cc:1151] Deleting tablet accb2dedc30e494ebd22c33594cee661
0505 07:54:07.539499 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:07.623862 (+ 84363us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.784173 (+160311us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.896927 (+112754us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":924204}
I20260505 07:54:07.910224 2308 leader_election.cc:290] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:07.910612 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet accb2dedc30e494ebd22c33594cee661 (table default.Table_fa6246cc3ccd4d7082becde923f88828 [id=860ab29ce1a44ad1b2480dbdb2b40000]) successfully deleted
I20260505 07:54:07.911595 2137 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 37) took 1296 ms. Trace:
I20260505 07:54:07.912556 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b4e886bebc4ab6a3c95290d18b4229" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:07.913268 854 leader_election.cc:343] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.911746 2137 rpcz_store.cc:276] 0505 07:54:06.615547 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.615619 (+ 72us) service_pool.cc:224] Handling call
0505 07:54:07.910675 (+1295056us) ts_tablet_manager.cc:1151] Deleting tablet a9b7c6e9463342a5a73af12da4447f89
0505 07:54:07.910690 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:07.911546 (+ 856us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1294848}
W20260505 07:54:07.914132 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet a9b7c6e9463342a5a73af12da4447f89 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9b7c6e9463342a5a73af12da4447f89 already in progress: creating tablet
W20260505 07:54:07.915936 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet ae829daad64545e687a9528e5c38b9ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae829daad64545e687a9528e5c38b9ed already in progress: creating tablet
I20260505 07:54:07.917971 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba616702dc6040bb805bb3ffc8a8a871" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:07.920837 2342 tablet.cc:1724] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:07.922387 2137 tablet_replica.cc:333] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
W20260505 07:54:07.922492 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e6a5f823db6143319d743d363a77ea47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6a5f823db6143319d743d363a77ea47 already in progress: creating tablet
I20260505 07:54:07.910759 1181 tablet_bootstrap.cc:492] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:54:07.916927 991 consensus_peers.cc:597] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:07.926486 2280 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s user 0.011s sys 0.047s
W20260505 07:54:07.930136 2280 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s user 0.012s sys 0.047s
I20260505 07:54:07.939846 1181 tablet_bootstrap.cc:654] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:07.948902 1176 tablet_bootstrap.cc:492] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:07.949048 1176 ts_tablet_manager.cc:1403] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.325s user 0.012s sys 0.003s
I20260505 07:54:07.960325 854 leader_election.cc:304] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.960703 1909 raft_consensus.cc:2749] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:07.961081 854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 47824.4us
W20260505 07:54:07.961342 854 leader_election.cc:343] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.961515 854 leader_election.cc:304] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.961774 1909 raft_consensus.cc:2749] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:07.746893 2276 ts_tablet_manager.cc:1950] T accb2dedc30e494ebd22c33594cee661 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:07.929839 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36b33d265ba04e93993c7e0a4efea1a4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:07.962723 911 raft_consensus.cc:3060] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:07.931396 2334 tablet.cc:1724] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:07.931846 1172 tablet_bootstrap.cc:492] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:07.965006 1172 tablet_bootstrap.cc:654] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:07.966164 911 raft_consensus.cc:2468] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.967257 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 33) took 1124 ms. Trace:
I20260505 07:54:07.967397 2276 rpcz_store.cc:276] 0505 07:54:06.843007 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.037668 (+194661us) service_pool.cc:224] Handling call
0505 07:54:07.653666 (+615998us) ts_tablet_manager.cc:1151] Deleting tablet accb2dedc30e494ebd22c33594cee661
0505 07:54:07.653677 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:07.704667 (+ 50990us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.745914 (+ 41247us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:07.967241 (+221327us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":593503}
I20260505 07:54:07.968025 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet accb2dedc30e494ebd22c33594cee661 (table default.Table_fa6246cc3ccd4d7082becde923f88828 [id=860ab29ce1a44ad1b2480dbdb2b40000]) successfully deleted
I20260505 07:54:07.968515 2276 tablet_replica.cc:333] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:07.968816 2276 raft_consensus.cc:2243] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:07.977195 991 leader_election.cc:304] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:07.955758 916 raft_consensus.cc:2468] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:07.983806 2137 raft_consensus.cc:2243] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:07.986752 1176 raft_consensus.cc:359] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.986927 1176 raft_consensus.cc:385] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:07.986992 1176 raft_consensus.cc:740] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:07.987224 2137 pending_rounds.cc:70] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: Trying to abort 1 pending ops.
I20260505 07:54:07.987231 1176 consensus_queue.cc:260] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.987319 2137 pending_rounds.cc:77] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7282555485442564096 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "85d485ae2d1d4d76a6a5d988cec8291a" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_394242d5ee184ae2b22cc355a2160fb9" dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" new_extra_config { } }
W20260505 07:54:07.987788 2137 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55776: Aborted: Op aborted
I20260505 07:54:07.987910 1176 ts_tablet_manager.cc:1434] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.039s user 0.002s sys 0.000s
I20260505 07:54:07.988181 1176 tablet_bootstrap.cc:492] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:07.988356 2137 raft_consensus.cc:2272] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:07.988299 1315 catalog_manager.cc:5081] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): alter failed for tablet 85d485ae2d1d4d76a6a5d988cec8291a (table default.Table_394242d5ee184ae2b22cc355a2160fb9 [id=6e71d3f928fc45c191f64a945980a107]): Aborted: Op aborted
I20260505 07:54:07.989451 2137 ts_tablet_manager.cc:1916] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:07.992462 1172 tablet_bootstrap.cc:492] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:07.992604 1172 ts_tablet_manager.cc:1403] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.061s user 0.001s sys 0.008s
I20260505 07:54:07.993506 1172 raft_consensus.cc:359] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.993673 1172 raft_consensus.cc:385] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:07.993764 1172 raft_consensus.cc:740] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:54:07.993821 854 leader_election.cc:343] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.993906 854 leader_election.cc:304] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.993939 1172 consensus_queue.cc:260] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:07.994091 1909 raft_consensus.cc:2749] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:07.994374 1172 ts_tablet_manager.cc:1434] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
W20260505 07:54:07.994490 854 leader_election.cc:343] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.994638 1172 tablet_bootstrap.cc:492] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:07.994658 854 leader_election.cc:304] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.994922 2284 raft_consensus.cc:2749] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:07.995289 854 leader_election.cc:343] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:07.995436 854 leader_election.cc:304] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:07.995674 2276 raft_consensus.cc:2272] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:07.996104 1172 tablet_bootstrap.cc:654] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.004781 1878 raft_consensus.cc:2804] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:08.005833 2284 raft_consensus.cc:2749] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:08.005837 991 leader_election.cc:304] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:08.006821 1878 raft_consensus.cc:697] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:08.007258 2334 tablet.cc:1724] T b404e50ae092419c9396772b3b14eeaa P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
W20260505 07:54:08.008097 854 leader_election.cc:343] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.009686 2307 raft_consensus.cc:2804] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:08.010061 1878 consensus_queue.cc:237] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.015690 854 leader_election.cc:304] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:08.015967 2284 raft_consensus.cc:2749] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:08.015892 967 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
W20260505 07:54:08.017594 854 leader_election.cc:343] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.008790 2276 ts_tablet_manager.cc:1916] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:08.020897 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21480c42b73e4bf0ab2453b085a867c0" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.020926 1172 tablet_bootstrap.cc:492] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.021478 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3fc8a9d9f04636b1f31bb88411c1da" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.022485 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de1a667b14fe494c8ee4be0f47a340be" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.023036 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d622612bdc404727a27b83769bf39738" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:08.023188 776 raft_consensus.cc:3060] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:08.024219 1176 tablet_bootstrap.cc:654] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.024384 2307 raft_consensus.cc:697] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:08.024678 2307 consensus_queue.cc:237] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:08.026237 776 raft_consensus.cc:2468] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:08.026276 1315 catalog_manager.cc:5075] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): alter failed for tablet 85d485ae2d1d4d76a6a5d988cec8291a (table default.Table_394242d5ee184ae2b22cc355a2160fb9 [id=6e71d3f928fc45c191f64a945980a107]),no further retry: Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:54:08.021960 1172 ts_tablet_manager.cc:1403] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.000s
I20260505 07:54:08.027006 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "934cea6666b1420eb12fe4510d2466fe" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:08.027266 776 raft_consensus.cc:3060] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:08.030630 776 raft_consensus.cc:2468] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:08.031682 854 leader_election.cc:304] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:08.031971 2284 raft_consensus.cc:2749] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:08.032958 854 leader_election.cc:343] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.033121 854 leader_election.cc:304] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:08.033411 2284 raft_consensus.cc:2749] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:08.033818 854 leader_election.cc:343] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.034004 854 leader_election.cc:304] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:08.033376 1172 raft_consensus.cc:359] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.035002 1172 raft_consensus.cc:385] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.036423 1172 raft_consensus.cc:740] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.037434 1172 consensus_queue.cc:260] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.045545 1176 tablet_bootstrap.cc:492] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:08.045689 1176 ts_tablet_manager.cc:1403] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.000s
I20260505 07:54:08.046430 1176 raft_consensus.cc:359] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.046586 1176 raft_consensus.cc:385] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.046643 1176 raft_consensus.cc:740] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.046801 1176 consensus_queue.cc:260] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.047267 1176 ts_tablet_manager.cc:1434] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:08.048132 1181 tablet_bootstrap.cc:492] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:08.048333 1181 ts_tablet_manager.cc:1403] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.148s user 0.009s sys 0.006s
I20260505 07:54:08.035528 2371 tablet.cc:1724] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:08.049914 2284 raft_consensus.cc:2749] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:08.050323 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "46274a44a06b4306b5dac43c66c85a80" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:08.050390 854 leader_election.cc:343] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.050892 1172 ts_tablet_manager.cc:1434] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.024s user 0.001s sys 0.001s
I20260505 07:54:08.051347 1172 tablet_bootstrap.cc:492] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.051985 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "297537a92684425d8508f7b3b4c9d379" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.052124 778 raft_consensus.cc:2468] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:08.052506 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29ba55a846548679cb3b86305d0d6ed" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.052868 1172 tablet_bootstrap.cc:654] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.052987 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f3544e74d941d286a2a0317b941ab6" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:08.053282 2342 tablet.cc:1724] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:08.050508 854 leader_election.cc:304] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:08.055315 2284 raft_consensus.cc:2749] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:08.060252 854 leader_election.cc:343] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.063282 854 leader_election.cc:304] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:08.063768 1181 raft_consensus.cc:359] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.064033 1181 raft_consensus.cc:385] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.064332 1181 raft_consensus.cc:740] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.064790 1181 consensus_queue.cc:260] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.066256 1181 ts_tablet_manager.cc:1434] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.017s user 0.000s sys 0.002s
I20260505 07:54:08.066998 2373 tablet.cc:1724] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:08.067992 2284 raft_consensus.cc:2749] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:08.068058 1181 tablet_bootstrap.cc:492] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:08.068770 2383 tablet.cc:1724] T c822fafe72544ca99fde910c8e764755 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:08.070511 1172 tablet_bootstrap.cc:492] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.071183 1181 tablet_bootstrap.cc:654] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.071220 1172 ts_tablet_manager.cc:1403] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.000s
I20260505 07:54:08.075142 2137 ts_tablet_manager.cc:1929] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:08.063026 2276 ts_tablet_manager.cc:1929] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:08.079694 2276 log.cc:1199] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/4662d4a3d88d4c2883a35048d6e11eb9
I20260505 07:54:08.080778 2276 ts_tablet_manager.cc:1950] T 4662d4a3d88d4c2883a35048d6e11eb9 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:08.084254 2371 tablet.cc:1724] T b404e50ae092419c9396772b3b14eeaa P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260505 07:54:08.089779 2377 tablet.cc:1724] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:08.103545 2268 ts_tablet_manager.cc:1929] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:08.105427 2325 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47472 (request call id 5) took 1316 ms. Trace:
I20260505 07:54:08.105665 1355 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 25072055f5534078adf5b5025614f951
I20260505 07:54:08.106160 1355 catalog_manager.cc:6741] ReplaceTablet: tablet 25072055f5534078adf5b5025614f951 deleted and replaced by tablet 52e2db0240e04d6d8e948ebaedcc7d24
I20260505 07:54:08.106702 2268 log.cc:1199] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/ccdde3e5a13e436d9f2328831e155adb
I20260505 07:54:08.107018 2325 rpcz_store.cc:276] 0505 07:54:06.763719 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.763763 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:08.079869 (+1316106us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:06.764209 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0505 07:54:06.764301 (+ 92us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0505 07:54:06.764311 (+ 10us) alter_schema_op.cc:66] Acquired schema lock
0505 07:54:06.764317 (+ 6us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0505 07:54:06.764350 (+ 33us) alter_schema_op.cc:128] START. Timestamp: 1777967646764280
0505 07:54:06.764353 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:06.764521 (+ 168us) log.cc:844] Serialized 173 byte log entry
0505 07:54:07.673405 (+908884us) op_driver.cc:464] REPLICATION: finished
0505 07:54:07.847884 (+174479us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0505 07:54:07.886556 (+ 38672us) log.cc:844] Serialized 5 byte log entry
0505 07:54:08.079319 (+192763us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.079503 (+ 184us) log.cc:844] Serialized 14 byte log entry
0505 07:54:08.079824 (+ 321us) alter_schema_op.cc:73] Released schema lock
0505 07:54:08.079829 (+ 5us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":174404,"prepare.queue_time_us":53,"prepare.run_cpu_time_us":7368,"prepare.run_wall_time_us":105123,"raft.queue_time_us":472,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":362,"replication_time_us":909011,"thread_start_us":448095,"threads_started":3,"wal-append.queue_time_us":238052,"wal-append.run_cpu_time_us":516,"wal-append.run_wall_time_us":1000564}]]}
I20260505 07:54:08.109532 2325 tablet.cc:1724] T c822fafe72544ca99fde910c8e764755 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:08.111047 2268 ts_tablet_manager.cc:1950] T ccdde3e5a13e436d9f2328831e155adb P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:08.112110 1028 tablet_service.cc:1558] Processing DeleteTablet for tablet 25072055f5534078adf5b5025614f951 with delete_type TABLET_DATA_DELETED (replaced by tablet 52e2db0240e04d6d8e948ebaedcc7d24) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:08.112310 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 4662d4a3d88d4c2883a35048d6e11eb9 (table default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9 [id=3aa673bdfdb2432bb2174d4a520d7a9e]) successfully deleted
I20260505 07:54:08.112375 883 tablet_service.cc:1558] Processing DeleteTablet for tablet 25072055f5534078adf5b5025614f951 with delete_type TABLET_DATA_DELETED (replaced by tablet 52e2db0240e04d6d8e948ebaedcc7d24) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:08.114311 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 4f3e076892234c1494b1a5c13cbfd2e2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f3e076892234c1494b1a5c13cbfd2e2 already in progress: creating tablet
W20260505 07:54:08.119504 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 09b46ff0101a4e529077b9e30492ace7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 09b46ff0101a4e529077b9e30492ace7 already in progress: creating tablet
W20260505 07:54:08.122406 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 02e06d8815034d91986cef12964860ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02e06d8815034d91986cef12964860ea already in progress: creating tablet
W20260505 07:54:08.124831 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet f200abef13ac42698a9b96fc8af82ff0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f200abef13ac42698a9b96fc8af82ff0 already in progress: creating tablet
I20260505 07:54:08.080716 1176 tablet_bootstrap.cc:492] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:08.081045 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de1a667b14fe494c8ee4be0f47a340be" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:08.125226 775 raft_consensus.cc:3060] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:08.126734 1176 tablet_bootstrap.cc:654] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.081127 780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "036d306d592f41cb96814aa60e30551e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:08.081868 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29ba55a846548679cb3b86305d0d6ed" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:08.082016 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "297537a92684425d8508f7b3b4c9d379" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:08.127980 772 raft_consensus.cc:3060] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:08.130931 772 raft_consensus.cc:2468] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:08.082139 773 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7001482d90364dbdb8b9c5bfb98e1cd4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.082247 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3fc8a9d9f04636b1f31bb88411c1da" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:08.083170 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21480c42b73e4bf0ab2453b085a867c0" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:08.100159 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36b33d265ba04e93993c7e0a4efea1a4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:08.132004 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e7887c3462f74740b314e3bec36e280d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7887c3462f74740b314e3bec36e280d already in progress: creating tablet
I20260505 07:54:08.132296 775 raft_consensus.cc:2468] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:08.100445 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7001482d90364dbdb8b9c5bfb98e1cd4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:08.103996 717 consensus_peers.cc:597] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aab3850799254dda9716fd5bf9404159. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:08.113947 2385 tablet.cc:1724] T 8d1ce00e487d4702a6affcad57a7fbe1 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:08.133056 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 25072055f5534078adf5b5025614f951 with delete_type TABLET_DATA_DELETED (replaced by tablet 52e2db0240e04d6d8e948ebaedcc7d24) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:08.133529 883 tablet_service.cc:1511] Processing CreateTablet for tablet 93b546377fd84568aa0f1bf5271b043f (DEFAULT_TABLE table=default.Table_82a1a0d4879f449d92ddc9df6749f98f [id=ba7529887fd64d89866527d8e0533a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.133903 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93b546377fd84568aa0f1bf5271b043f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.134739 2137 log.cc:1199] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/85d485ae2d1d4d76a6a5d988cec8291a
I20260505 07:54:08.135097 2137 ts_tablet_manager.cc:1950] T 85d485ae2d1d4d76a6a5d988cec8291a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:08.135360 2137 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 40) took 1197 ms. Trace:
I20260505 07:54:08.135421 2137 rpcz_store.cc:276] 0505 07:54:06.938754 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.938800 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:07.922346 (+983546us) ts_tablet_manager.cc:1151] Deleting tablet 85d485ae2d1d4d76a6a5d988cec8291a
0505 07:54:07.922355 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:07.984127 (+ 61772us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x616000183454. stack: 00007fb70c947086 00007fb70c946e40 00007fb70b9003ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fb71888a7e0 00007fb719bde283 00007fb71c7a3a77 00007fb71c7d6b44 00007fb71c7b7560 00007fb71c7b72ed 0000000000aa4cac 00007fb70c98cf6f 00007fb70c99028b 00007fb70c990108
0505 07:54:07.987154 (+ 3027us) spinlock_profiling.cc:243] Waited 2.6 ms on lock 0x60f0002a4670. stack: 00007fb70c947086 00007fb70c946e40 00007fb70b9003ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fb71881a492 00007fb71888a8c3 00007fb719bde283 00007fb71c7a3a77 00007fb71c7d6b44 00007fb71c7b7560 00007fb71c7b72ed 0000000000aa4cac 00007fb70c98cf6f 00007fb70c99028b
0505 07:54:07.992891 (+ 5737us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.075091 (+ 82200us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.135348 (+ 60257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146541568,"tablet-delete.queue_time_us":937203}
I20260505 07:54:08.140858 2284 raft_consensus.cc:493] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:08.140923 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 6) took 1355 ms. Trace:
I20260505 07:54:08.140974 2284 raft_consensus.cc:515] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.142037 2268 rpcz_store.cc:276] 0505 07:54:06.772612 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.772663 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:07.823061 (+1050398us) ts_tablet_manager.cc:1151] Deleting tablet ccdde3e5a13e436d9f2328831e155adb
0505 07:54:07.823069 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:08.005264 (+182195us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.075264 (+ 70000us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.128266 (+ 53002us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1050141}
I20260505 07:54:08.142345 2268 tablet_replica.cc:333] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:08.142685 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet ccdde3e5a13e436d9f2328831e155adb (table default.Table_efeac090fb094bdf82f547af366d562c [id=047ae44eaf5e48ba8b11e90f6d5f8119]) successfully deleted
I20260505 07:54:08.142841 2268 raft_consensus.cc:2243] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:08.143121 2268 raft_consensus.cc:2272] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:08.142953 2276 tablet_replica.cc:333] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:08.144749 2268 ts_tablet_manager.cc:1916] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:08.145617 2276 raft_consensus.cc:2243] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:08.147946 1909 raft_consensus.cc:493] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:08.148059 1909 raft_consensus.cc:515] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.149159 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 93b546377fd84568aa0f1bf5271b043f (DEFAULT_TABLE table=default.Table_82a1a0d4879f449d92ddc9df6749f98f [id=ba7529887fd64d89866527d8e0533a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.149513 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93b546377fd84568aa0f1bf5271b043f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.152557 1018 tablet_service.cc:1511] Processing CreateTablet for tablet 56b8b2d40a034fa49323f6b394b49509 (DEFAULT_TABLE table=default.Table_f99f3923bf7742a6a16cb971a60c1245 [id=094ff08876f7499789f28d67f06eb3d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.152921 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b8b2d40a034fa49323f6b394b49509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.153496 1181 tablet_bootstrap.cc:492] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:08.153609 2284 leader_election.cc:290] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:08.153811 883 tablet_service.cc:1511] Processing CreateTablet for tablet 56b8b2d40a034fa49323f6b394b49509 (DEFAULT_TABLE table=default.Table_f99f3923bf7742a6a16cb971a60c1245 [id=094ff08876f7499789f28d67f06eb3d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.154153 883 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b8b2d40a034fa49323f6b394b49509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.153895 1909 leader_election.cc:290] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:08.155109 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 91d57f5bb6e847e6b099134d1382b730 (DEFAULT_TABLE table=default.Table_6bf0ac886e27494d867dd60c897ba88f [id=1c28170e4afa4187840ff16eb3015044]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.146488 2276 raft_consensus.cc:2272] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:08.155812 890 tablet_service.cc:1511] Processing CreateTablet for tablet 91d57f5bb6e847e6b099134d1382b730 (DEFAULT_TABLE table=default.Table_6bf0ac886e27494d867dd60c897ba88f [id=1c28170e4afa4187840ff16eb3015044]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.156008 897 tablet_service.cc:1511] Processing CreateTablet for tablet 64ca343bd32a4eeb80901bc3157b3b6e (DEFAULT_TABLE table=default.Table_497d4c8424b841d38672247e2b4e33f2 [id=ce5a935268d844f4878670e02bff2dd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.156117 890 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d57f5bb6e847e6b099134d1382b730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.142268 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54e4f7f895d943639653c8c32724baf2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.147220 762 tablet_service.cc:1511] Processing CreateTablet for tablet 93b546377fd84568aa0f1bf5271b043f (DEFAULT_TABLE table=default.Table_82a1a0d4879f449d92ddc9df6749f98f [id=ba7529887fd64d89866527d8e0533a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.156627 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93b546377fd84568aa0f1bf5271b043f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:08.157423 988 leader_election.cc:343] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:08.157723 988 leader_election.cc:343] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:08.157976 988 leader_election.cc:343] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:08.158561 988 leader_election.cc:343] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:08.159475 988 leader_election.cc:343] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.160440 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df1b62a30b2842b897a04c793bb5e7ce" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:08.160832 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d57f5bb6e847e6b099134d1382b730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.161252 1181 ts_tablet_manager.cc:1403] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.087s user 0.001s sys 0.006s
I20260505 07:54:08.161288 2276 ts_tablet_manager.cc:1916] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:08.161672 988 leader_election.cc:343] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:08.162094 856 leader_election.cc:343] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.148825 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df1b62a30b2842b897a04c793bb5e7ce" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.149425 747 tablet_service.cc:1511] Processing CreateTablet for tablet 56b8b2d40a034fa49323f6b394b49509 (DEFAULT_TABLE table=default.Table_f99f3923bf7742a6a16cb971a60c1245 [id=094ff08876f7499789f28d67f06eb3d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.162874 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56b8b2d40a034fa49323f6b394b49509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.151180 757 tablet_service.cc:1511] Processing CreateTablet for tablet 91d57f5bb6e847e6b099134d1382b730 (DEFAULT_TABLE table=default.Table_6bf0ac886e27494d867dd60c897ba88f [id=1c28170e4afa4187840ff16eb3015044]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.166260 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d57f5bb6e847e6b099134d1382b730. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.167698 897 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ca343bd32a4eeb80901bc3157b3b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.152065 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "36b33d265ba04e93993c7e0a4efea1a4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:08.153478 751 tablet_service.cc:1511] Processing CreateTablet for tablet 64ca343bd32a4eeb80901bc3157b3b6e (DEFAULT_TABLE table=default.Table_497d4c8424b841d38672247e2b4e33f2 [id=ce5a935268d844f4878670e02bff2dd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.170991 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 64ca343bd32a4eeb80901bc3157b3b6e (DEFAULT_TABLE table=default.Table_497d4c8424b841d38672247e2b4e33f2 [id=ce5a935268d844f4878670e02bff2dd3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.171248 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ca343bd32a4eeb80901bc3157b3b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.175693 1019 tablet_service.cc:1511] Processing CreateTablet for tablet 5b7a1da5cbc94df9b3b697251087b59a (DEFAULT_TABLE table=default.Table_85003b58445a4d7fa3848473b7a93d08 [id=e47c1f5f60e647748d158c61c9e75f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.176208 758 tablet_service.cc:1511] Processing CreateTablet for tablet 5b7a1da5cbc94df9b3b697251087b59a (DEFAULT_TABLE table=default.Table_85003b58445a4d7fa3848473b7a93d08 [id=e47c1f5f60e647748d158c61c9e75f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.176517 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b7a1da5cbc94df9b3b697251087b59a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.176975 1016 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4342c2c6d142ad90b0270608007eda (DEFAULT_TABLE table=default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.178246 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 090fbe6c480249338e7210050770adeb (DEFAULT_TABLE table=default.Table_1a1a26d7818c4e038e590867f615d13f [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.179174 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:08.180105 750 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4342c2c6d142ad90b0270608007eda (DEFAULT_TABLE table=default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.180426 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4342c2c6d142ad90b0270608007eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.180464 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:08.181597 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:08.182323 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:08.183307 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:08.183804 761 tablet_service.cc:1511] Processing CreateTablet for tablet 090fbe6c480249338e7210050770adeb (DEFAULT_TABLE table=default.Table_1a1a26d7818c4e038e590867f615d13f [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.183902 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54e4f7f895d943639653c8c32724baf2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:08.183936 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:08.184093 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 090fbe6c480249338e7210050770adeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.184633 2268 ts_tablet_manager.cc:1929] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:08.184746 2268 log.cc:1199] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/976359601ad74b149696dc7f555b550c
I20260505 07:54:08.184724 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:08.185076 2268 ts_tablet_manager.cc:1950] T 976359601ad74b149696dc7f555b550c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:08.185550 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet df3a49b59ed6485f970074391f932482 (table default.Table_7af623f581af498b86c8c3b1adbc6b80 [id=306149ae47a745edabf863d5004d5012]). Expected version 1 got 0
I20260505 07:54:08.186343 1181 raft_consensus.cc:359] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.186518 1181 raft_consensus.cc:385] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.186476 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:08.187819 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:08.188688 746 tablet_service.cc:1511] Processing CreateTablet for tablet 152176572c904b8296073afad0585817 (DEFAULT_TABLE table=default.Table_fbe275b2392f4ab7a874b451ad147cb3 [id=ecbb522ec2ce4de2946f5196b2031262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.189057 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 152176572c904b8296073afad0585817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.190807 749 tablet_service.cc:1511] Processing CreateTablet for tablet 1f043cd9042f4e1180d7c0cd90713b03 (DEFAULT_TABLE table=default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.190876 756 tablet_service.cc:1511] Processing CreateTablet for tablet 35cd0e809d6345a6a9825d4dc44d7575 (DEFAULT_TABLE table=default.Table_b5f852a0e6f44abea12de59734ee34aa [id=0cfb0fb3f7914164a2ba70e445b9cfbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.191120 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f043cd9042f4e1180d7c0cd90713b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.191377 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35cd0e809d6345a6a9825d4dc44d7575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.191776 760 tablet_service.cc:1511] Processing CreateTablet for tablet 4941b1c1f7604fb7978b41790a0e050a (DEFAULT_TABLE table=default.Table_02e4c2a300de46c897a536456185a75b [id=373438927ff94feda84babf40c0426a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.191895 752 tablet_service.cc:1511] Processing CreateTablet for tablet 0d86424d75b4491d8967ee0c0203b5ca (DEFAULT_TABLE table=default.Table_102bde0a26724f2bb7fbd86f0683813e [id=46f1327e89b7402c94e9d443e37a01a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.192093 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4941b1c1f7604fb7978b41790a0e050a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.192239 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d86424d75b4491d8967ee0c0203b5ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.192677 759 tablet_service.cc:1511] Processing CreateTablet for tablet e227bcd23048402cb0412ef107b766a4 (DEFAULT_TABLE table=default.Table_37117a0adb9a48248e0a53cffbf66e9c [id=87805bed8b344d0786542094ac1ae562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.192998 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e227bcd23048402cb0412ef107b766a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.187011 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 152176572c904b8296073afad0585817 (DEFAULT_TABLE table=default.Table_fbe275b2392f4ab7a874b451ad147cb3 [id=ecbb522ec2ce4de2946f5196b2031262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.190246 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 0d86424d75b4491d8967ee0c0203b5ca (DEFAULT_TABLE table=default.Table_102bde0a26724f2bb7fbd86f0683813e [id=46f1327e89b7402c94e9d443e37a01a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.190531 1029 tablet_service.cc:1511] Processing CreateTablet for tablet 1f043cd9042f4e1180d7c0cd90713b03 (DEFAULT_TABLE table=default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.191182 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 35cd0e809d6345a6a9825d4dc44d7575 (DEFAULT_TABLE table=default.Table_b5f852a0e6f44abea12de59734ee34aa [id=0cfb0fb3f7914164a2ba70e445b9cfbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.196194 1021 tablet_service.cc:1511] Processing CreateTablet for tablet 4941b1c1f7604fb7978b41790a0e050a (DEFAULT_TABLE table=default.Table_02e4c2a300de46c897a536456185a75b [id=373438927ff94feda84babf40c0426a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.196475 1017 tablet_service.cc:1511] Processing CreateTablet for tablet a5fa390b240d4079b8ada7c20adc33a4 (DEFAULT_TABLE table=default.Table_0251d9bbbee946e9a6f8757aefc090d2 [id=7355dc2d5f284f8b99d16d03d7d8e65a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.196543 1020 tablet_service.cc:1511] Processing CreateTablet for tablet e227bcd23048402cb0412ef107b766a4 (DEFAULT_TABLE table=default.Table_37117a0adb9a48248e0a53cffbf66e9c [id=87805bed8b344d0786542094ac1ae562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.196759 1033 tablet_service.cc:1511] Processing CreateTablet for tablet 490483562d8f4c2daa7eaa333a9f3a0c (DEFAULT_TABLE table=default.Table_6d0c4c13f95a4af2a119861b7923d94a [id=fe7ee6e8118840e8a9f70a007d80c2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.196933 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 11bdeb785eb34110be29f54fb835f132 (DEFAULT_TABLE table=default.Table_94e146286fd044179da20696e871193c [id=7bce4b74181644ca9529c778f9851564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.186582 1181 raft_consensus.cc:740] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.199856 1181 consensus_queue.cc:260] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.200420 1181 ts_tablet_manager.cc:1434] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.039s user 0.002s sys 0.000s
I20260505 07:54:08.200947 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64ca343bd32a4eeb80901bc3157b3b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.204035 1035 tablet_service.cc:1511] Processing CreateTablet for tablet e47c1a8889644b729f32c0a5563b71ad (DEFAULT_TABLE table=default.Table_cf6508cc6f1e49d5b6971e4f032fa721 [id=dca517b1e5054077998df758811f6df9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.204363 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47c1a8889644b729f32c0a5563b71ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.207512 754 tablet_service.cc:1511] Processing CreateTablet for tablet 0b79355b53e24bf2b307e1dd4b35a6a9 (DEFAULT_TABLE table=default.Table_8fef0056e563428d855e8e9f8053c0cb [id=11a85aac41cb4d03aa7b507c817b7a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.207564 752 tablet_service.cc:1511] Processing CreateTablet for tablet 11bdeb785eb34110be29f54fb835f132 (DEFAULT_TABLE table=default.Table_94e146286fd044179da20696e871193c [id=7bce4b74181644ca9529c778f9851564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.207893 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b79355b53e24bf2b307e1dd4b35a6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.208271 755 tablet_service.cc:1511] Processing CreateTablet for tablet db430ab084354b48aceb445ff3306ef9 (DEFAULT_TABLE table=default.Table_1e40e3f38bf54ca28a64fa4b0c46a123 [id=ac2132cb9b1a410eaee6793aa6b1bbf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.208539 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db430ab084354b48aceb445ff3306ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.210497 1022 tablet_service.cc:1511] Processing CreateTablet for tablet 0b79355b53e24bf2b307e1dd4b35a6a9 (DEFAULT_TABLE table=default.Table_8fef0056e563428d855e8e9f8053c0cb [id=11a85aac41cb4d03aa7b507c817b7a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.210880 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b79355b53e24bf2b307e1dd4b35a6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.211946 743 tablet_service.cc:1511] Processing CreateTablet for tablet dfbe4dff192741099be5ab10a96adeb7 (DEFAULT_TABLE table=default.Table_6d46a7a584f44bfbb6910845bb4b53a2 [id=f74147100fd54974ae4c67328b7023e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.212256 743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfbe4dff192741099be5ab10a96adeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.207617 746 tablet_service.cc:1511] Processing CreateTablet for tablet a5fa390b240d4079b8ada7c20adc33a4 (DEFAULT_TABLE table=default.Table_0251d9bbbee946e9a6f8757aefc090d2 [id=7355dc2d5f284f8b99d16d03d7d8e65a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.214210 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5fa390b240d4079b8ada7c20adc33a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.214484 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11bdeb785eb34110be29f54fb835f132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.215916 1034 tablet_service.cc:1511] Processing CreateTablet for tablet db430ab084354b48aceb445ff3306ef9 (DEFAULT_TABLE table=default.Table_1e40e3f38bf54ca28a64fa4b0c46a123 [id=ac2132cb9b1a410eaee6793aa6b1bbf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.216238 753 tablet_service.cc:1511] Processing CreateTablet for tablet a32eeaca77c1475f879465cb1046543b (DEFAULT_TABLE table=default.Table_394242d5ee184ae2b22cc355a2160fb9 [id=6e71d3f928fc45c191f64a945980a107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.216543 753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a32eeaca77c1475f879465cb1046543b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.216699 1030 tablet_service.cc:1511] Processing CreateTablet for tablet dfbe4dff192741099be5ab10a96adeb7 (DEFAULT_TABLE table=default.Table_6d46a7a584f44bfbb6910845bb4b53a2 [id=f74147100fd54974ae4c67328b7023e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.217245 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db430ab084354b48aceb445ff3306ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.219956 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:08.221532 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:08.223174 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfbe4dff192741099be5ab10a96adeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.222975 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:08.224287 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:08.225889 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:08.230396 744 tablet_service.cc:1511] Processing CreateTablet for tablet 13800db6031045babf951adbee263377 (DEFAULT_TABLE table=default.Table_778f98abad284a74910d0d8c8e1d9a58 [id=5be875bc42a146bb86bb46462b12a96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.230736 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13800db6031045babf951adbee263377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.233570 1035 tablet_service.cc:1511] Processing CreateTablet for tablet 38eee32a6fd84f89a1e88074f16d4242 (DEFAULT_TABLE table=default.Table_d0e52326b0e041e3941fd45dab79f2da [id=4231aa55ac4e481eb686252d30d91195]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.233892 1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38eee32a6fd84f89a1e88074f16d4242. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:08.235268 856 leader_election.cc:343] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.235682 1909 raft_consensus.cc:493] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:08.235813 1909 raft_consensus.cc:515] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.237713 1022 tablet_service.cc:1511] Processing CreateTablet for tablet a32eeaca77c1475f879465cb1046543b (DEFAULT_TABLE table=default.Table_394242d5ee184ae2b22cc355a2160fb9 [id=6e71d3f928fc45c191f64a945980a107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.238107 1022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a32eeaca77c1475f879465cb1046543b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.238065 747 tablet_service.cc:1511] Processing CreateTablet for tablet 90612f0afb4949f491e625c61e40d247 (DEFAULT_TABLE table=default.Table_359f92a4ff554e3e80468261b0e86966 [id=9930805fdf364f25aea11f859b50bc71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.238761 747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90612f0afb4949f491e625c61e40d247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.241127 1030 tablet_service.cc:1511] Processing CreateTablet for tablet 90612f0afb4949f491e625c61e40d247 (DEFAULT_TABLE table=default.Table_359f92a4ff554e3e80468261b0e86966 [id=9930805fdf364f25aea11f859b50bc71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.238433 755 tablet_service.cc:1511] Processing CreateTablet for tablet 4c0fea885db74bf4be31d464e90e2292 (DEFAULT_TABLE table=default.Table_830cdb08572d46bda135fc765659e6d6 [id=c3cbc16af9cb493593014e13ed67004e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.241510 1030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90612f0afb4949f491e625c61e40d247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.241546 755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0fea885db74bf4be31d464e90e2292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.245905 885 tablet_service.cc:1511] Processing CreateTablet for tablet 5b7a1da5cbc94df9b3b697251087b59a (DEFAULT_TABLE table=default.Table_85003b58445a4d7fa3848473b7a93d08 [id=e47c1f5f60e647748d158c61c9e75f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.246263 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b7a1da5cbc94df9b3b697251087b59a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.247308 893 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4342c2c6d142ad90b0270608007eda (DEFAULT_TABLE table=default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.247646 893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4342c2c6d142ad90b0270608007eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.247887 759 tablet_service.cc:1511] Processing CreateTablet for tablet 490483562d8f4c2daa7eaa333a9f3a0c (DEFAULT_TABLE table=default.Table_6d0c4c13f95a4af2a119861b7923d94a [id=fe7ee6e8118840e8a9f70a007d80c2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.248584 1909 leader_election.cc:290] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:08.248574 754 tablet_service.cc:1511] Processing CreateTablet for tablet 38eee32a6fd84f89a1e88074f16d4242 (DEFAULT_TABLE table=default.Table_d0e52326b0e041e3941fd45dab79f2da [id=4231aa55ac4e481eb686252d30d91195]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.248024 756 tablet_service.cc:1511] Processing CreateTablet for tablet e47c1a8889644b729f32c0a5563b71ad (DEFAULT_TABLE table=default.Table_cf6508cc6f1e49d5b6971e4f032fa721 [id=dca517b1e5054077998df758811f6df9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.249105 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38eee32a6fd84f89a1e88074f16d4242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.249332 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47c1a8889644b729f32c0a5563b71ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.250952 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490483562d8f4c2daa7eaa333a9f3a0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.252091 899 tablet_service.cc:1511] Processing CreateTablet for tablet 090fbe6c480249338e7210050770adeb (DEFAULT_TABLE table=default.Table_1a1a26d7818c4e038e590867f615d13f [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.252417 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 090fbe6c480249338e7210050770adeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.257853 894 tablet_service.cc:1511] Processing CreateTablet for tablet 152176572c904b8296073afad0585817 (DEFAULT_TABLE table=default.Table_fbe275b2392f4ab7a874b451ad147cb3 [id=ecbb522ec2ce4de2946f5196b2031262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.258170 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 152176572c904b8296073afad0585817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.258478 2137 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 39) took 1323 ms. Trace:
I20260505 07:54:08.258555 2137 rpcz_store.cc:276] 0505 07:54:06.935372 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.935420 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:08.258365 (+1322945us) ts_tablet_manager.cc:1151] Deleting tablet b12d2d2b866348d781209fa787da7d7e
0505 07:54:08.258374 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:08.258461 (+ 87us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1252491}
I20260505 07:54:08.258852 2137 tablet_replica.cc:333] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:08.259027 2137 raft_consensus.cc:2243] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:08.259179 2137 raft_consensus.cc:2272] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:08.259728 2137 ts_tablet_manager.cc:1916] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:08.261015 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b7a1da5cbc94df9b3b697251087b59a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.270258 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 7) took 1480 ms. Trace:
I20260505 07:54:08.270375 2268 rpcz_store.cc:276] 0505 07:54:06.781995 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.782043 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:08.142295 (+1360252us) ts_tablet_manager.cc:1151] Deleting tablet 976359601ad74b149696dc7f555b550c
0505 07:54:08.142303 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:08.149058 (+ 6755us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.184569 (+ 35511us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.262331 (+ 77762us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1360045}
I20260505 07:54:08.271076 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490483562d8f4c2daa7eaa333a9f3a0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.271889 886 tablet_service.cc:1511] Processing CreateTablet for tablet 0d86424d75b4491d8967ee0c0203b5ca (DEFAULT_TABLE table=default.Table_102bde0a26724f2bb7fbd86f0683813e [id=46f1327e89b7402c94e9d443e37a01a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.272264 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d86424d75b4491d8967ee0c0203b5ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.283780 2201 consensus_queue.cc:1048] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:08.285490 900 tablet_service.cc:1511] Processing CreateTablet for tablet 1f043cd9042f4e1180d7c0cd90713b03 (DEFAULT_TABLE table=default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.285805 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f043cd9042f4e1180d7c0cd90713b03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.287810 887 tablet_service.cc:1511] Processing CreateTablet for tablet 35cd0e809d6345a6a9825d4dc44d7575 (DEFAULT_TABLE table=default.Table_b5f852a0e6f44abea12de59734ee34aa [id=0cfb0fb3f7914164a2ba70e445b9cfbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.288126 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35cd0e809d6345a6a9825d4dc44d7575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.288645 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4342c2c6d142ad90b0270608007eda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.289088 1878 consensus_queue.cc:1048] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260505 07:54:08.291394 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 4c0fea885db74bf4be31d464e90e2292 (DEFAULT_TABLE table=default.Table_830cdb08572d46bda135fc765659e6d6 [id=c3cbc16af9cb493593014e13ed67004e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.291833 898 tablet_service.cc:1511] Processing CreateTablet for tablet 490483562d8f4c2daa7eaa333a9f3a0c (DEFAULT_TABLE table=default.Table_6d0c4c13f95a4af2a119861b7923d94a [id=fe7ee6e8118840e8a9f70a007d80c2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.292116 898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490483562d8f4c2daa7eaa333a9f3a0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.292356 2105 consensus_queue.cc:1048] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260505 07:54:08.292419 882 tablet_service.cc:1511] Processing CreateTablet for tablet a5fa390b240d4079b8ada7c20adc33a4 (DEFAULT_TABLE table=default.Table_0251d9bbbee946e9a6f8757aefc090d2 [id=7355dc2d5f284f8b99d16d03d7d8e65a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.292661 882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5fa390b240d4079b8ada7c20adc33a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.293111 901 tablet_service.cc:1511] Processing CreateTablet for tablet 0b79355b53e24bf2b307e1dd4b35a6a9 (DEFAULT_TABLE table=default.Table_8fef0056e563428d855e8e9f8053c0cb [id=11a85aac41cb4d03aa7b507c817b7a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.293638 895 tablet_service.cc:1511] Processing CreateTablet for tablet 38eee32a6fd84f89a1e88074f16d4242 (DEFAULT_TABLE table=default.Table_d0e52326b0e041e3941fd45dab79f2da [id=4231aa55ac4e481eb686252d30d91195]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.295794 892 tablet_service.cc:1511] Processing CreateTablet for tablet e227bcd23048402cb0412ef107b766a4 (DEFAULT_TABLE table=default.Table_37117a0adb9a48248e0a53cffbf66e9c [id=87805bed8b344d0786542094ac1ae562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.296111 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e227bcd23048402cb0412ef107b766a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.297325 827 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:08.299810 888 tablet_service.cc:1511] Processing CreateTablet for tablet dfbe4dff192741099be5ab10a96adeb7 (DEFAULT_TABLE table=default.Table_6d46a7a584f44bfbb6910845bb4b53a2 [id=f74147100fd54974ae4c67328b7023e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.300139 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfbe4dff192741099be5ab10a96adeb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.303786 896 tablet_service.cc:1511] Processing CreateTablet for tablet 11bdeb785eb34110be29f54fb835f132 (DEFAULT_TABLE table=default.Table_94e146286fd044179da20696e871193c [id=7bce4b74181644ca9529c778f9851564]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.304107 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11bdeb785eb34110be29f54fb835f132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.307806 884 tablet_service.cc:1511] Processing CreateTablet for tablet 4941b1c1f7604fb7978b41790a0e050a (DEFAULT_TABLE table=default.Table_02e4c2a300de46c897a536456185a75b [id=373438927ff94feda84babf40c0426a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.308132 884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4941b1c1f7604fb7978b41790a0e050a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.311865 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b79355b53e24bf2b307e1dd4b35a6a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.313967 886 tablet_service.cc:1511] Processing CreateTablet for tablet 13800db6031045babf951adbee263377 (DEFAULT_TABLE table=default.Table_778f98abad284a74910d0d8c8e1d9a58 [id=5be875bc42a146bb86bb46462b12a96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.314312 886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13800db6031045babf951adbee263377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.316751 889 tablet_service.cc:1511] Processing CreateTablet for tablet e47c1a8889644b729f32c0a5563b71ad (DEFAULT_TABLE table=default.Table_cf6508cc6f1e49d5b6971e4f032fa721 [id=dca517b1e5054077998df758811f6df9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.316800 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "297537a92684425d8508f7b3b4c9d379" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:08.316857 894 tablet_service.cc:1511] Processing CreateTablet for tablet a32eeaca77c1475f879465cb1046543b (DEFAULT_TABLE table=default.Table_394242d5ee184ae2b22cc355a2160fb9 [id=6e71d3f928fc45c191f64a945980a107]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.293465 891 tablet_service.cc:1511] Processing CreateTablet for tablet db430ab084354b48aceb445ff3306ef9 (DEFAULT_TABLE table=default.Table_1e40e3f38bf54ca28a64fa4b0c46a123 [id=ac2132cb9b1a410eaee6793aa6b1bbf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.317030 913 raft_consensus.cc:2393] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:08.317135 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38eee32a6fd84f89a1e88074f16d4242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.317160 887 tablet_service.cc:1511] Processing CreateTablet for tablet 90612f0afb4949f491e625c61e40d247 (DEFAULT_TABLE table=default.Table_359f92a4ff554e3e80468261b0e86966 [id=9930805fdf364f25aea11f859b50bc71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.317441 900 tablet_service.cc:1511] Processing CreateTablet for tablet 4c0fea885db74bf4be31d464e90e2292 (DEFAULT_TABLE table=default.Table_830cdb08572d46bda135fc765659e6d6 [id=c3cbc16af9cb493593014e13ed67004e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.319780 889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e47c1a8889644b729f32c0a5563b71ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.320029 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 976359601ad74b149696dc7f555b550c (table default.Table_cb7435665bdd45ff8f517372a18d10b6 [id=0d5a336f16bb433fb2ee0f23189afc91]) successfully deleted
W20260505 07:54:08.320392 1315 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 74358.6us
I20260505 07:54:08.327734 2276 ts_tablet_manager.cc:1929] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:08.327975 2276 log.cc:1199] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/a9494c3b6a99487596a030856b371b68
I20260505 07:54:08.328944 2276 ts_tablet_manager.cc:1950] T a9494c3b6a99487596a030856b371b68 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:08.330727 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "934cea6666b1420eb12fe4510d2466fe" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:08.330943 913 raft_consensus.cc:2393] T 934cea6666b1420eb12fe4510d2466fe P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
W20260505 07:54:08.332157 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet b12d2d2b866348d781209fa787da7d7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b12d2d2b866348d781209fa787da7d7e already in progress: creating tablet
I20260505 07:54:08.332347 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:08.335651 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c822fafe72544ca99fde910c8e764755 (table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f]). Expected version 1 got 0
I20260505 07:54:08.336318 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:08.336557 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:08.337198 891 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db430ab084354b48aceb445ff3306ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.337067 1353 catalog_manager.cc:5671] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:08.337484 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 08c6ed755fc64080a2310fe9a4e0a68b (table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b]). Expected version 3 got 0
I20260505 07:54:08.337988 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:08.338654 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b406c4c6ba741bc85eef54c6f576459" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.338901 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
I20260505 07:54:08.339104 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:08.338747 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0fea885db74bf4be31d464e90e2292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.339291 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]). Expected version 2 got 0
I20260505 07:54:08.341678 1172 raft_consensus.cc:359] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.342904 1172 raft_consensus.cc:385] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.343397 1172 raft_consensus.cc:740] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.345218 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90612f0afb4949f491e625c61e40d247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.347018 2137 ts_tablet_manager.cc:1929] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:08.347129 2137 log.cc:1199] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/4662d4a3d88d4c2883a35048d6e11eb9
I20260505 07:54:08.347527 2137 ts_tablet_manager.cc:1950] T 4662d4a3d88d4c2883a35048d6e11eb9 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:08.351739 2137 tablet_replica.cc:333] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:08.351967 2137 raft_consensus.cc:2243] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:08.352246 2137 raft_consensus.cc:2272] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:08.353224 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 4662d4a3d88d4c2883a35048d6e11eb9 (table default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9 [id=3aa673bdfdb2432bb2174d4a520d7a9e]) successfully deleted
I20260505 07:54:08.354419 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 090fbe6c480249338e7210050770adeb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:08.354406 988 leader_election.cc:343] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.354635 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35cd0e809d6345a6a9825d4dc44d7575. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.354874 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d86424d75b4491d8967ee0c0203b5ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.355176 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4941b1c1f7604fb7978b41790a0e050a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.360072 2137 ts_tablet_manager.cc:1916] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:08.362663 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a32eeaca77c1475f879465cb1046543b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.363720 1176 tablet_bootstrap.cc:492] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:08.363873 1176 ts_tablet_manager.cc:1403] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.283s user 0.008s sys 0.001s
I20260505 07:54:08.365137 2276 tablet_replica.cc:333] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:08.366557 1172 consensus_queue.cc:260] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:08.368965 854 leader_election.cc:343] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.369092 854 leader_election.cc:304] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:08.369096 1172 ts_tablet_manager.cc:1434] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.059s user 0.003s sys 0.002s
I20260505 07:54:08.368827 2276 raft_consensus.cc:2243] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:08.354691 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 67045.3us
I20260505 07:54:08.369705 1172 tablet_bootstrap.cc:492] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.369752 1909 raft_consensus.cc:2706] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Received pre-election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260505 07:54:08.370260 2276 raft_consensus.cc:2272] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:08.372356 2201 consensus_queue.cc:1048] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:08.374884 988 leader_election.cc:343] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.375609 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 152176572c904b8296073afad0585817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.375836 1029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f043cd9042f4e1180d7c0cd90713b03. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:08.378585 988 leader_election.cc:343] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.380024 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 13800db6031045babf951adbee263377 (DEFAULT_TABLE table=default.Table_778f98abad284a74910d0d8c8e1d9a58 [id=5be875bc42a146bb86bb46462b12a96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.380756 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
W20260505 07:54:08.380777 988 leader_election.cc:343] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.381096 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet a9494c3b6a99487596a030856b371b68 (table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]) successfully deleted
W20260505 07:54:08.381218 988 leader_election.cc:343] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:08.381500 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 54e4f7f895d943639653c8c32724baf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54e4f7f895d943639653c8c32724baf2 already in progress: creating tablet
I20260505 07:54:08.381667 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
W20260505 07:54:08.381768 988 leader_election.cc:343] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.381985 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:08.382210 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:08.382419 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 42360f8ee4ff47c2af8d287f792207de (table default.Table_829cc346ac834637ab2e2198944a8550 [id=2c91868ddba84103b9a5c78201842a9e]). Expected version 1 got 0
I20260505 07:54:08.382928 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:08.383697 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]). Expected version 1 got 0
I20260505 07:54:08.384043 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:08.384907 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:08.385277 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]). Expected version 2 got 0
I20260505 07:54:08.364691 1176 raft_consensus.cc:359] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:08.385565 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:08.385735 1176 raft_consensus.cc:385] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.386263 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]). Expected version 2 got 0
I20260505 07:54:08.386299 1176 raft_consensus.cc:740] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.387072 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_95fae21889c84b61ad67610d209031b2 [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 1 got 0
I20260505 07:54:08.388140 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:08.388769 1176 consensus_queue.cc:260] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:08.390316 1353 catalog_manager.cc:5671] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:08.390920 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:08.391352 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_728dc87e78954a7fbbf495d74705e382 [id=2c06ff8eed31432e87f08e9b59049652]). Expected version 2 got 0
I20260505 07:54:08.391664 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:08.392148 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
I20260505 07:54:08.393217 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
W20260505 07:54:08.398360 988 leader_election.cc:343] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:08.398980 988 consensus_peers.cc:597] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: accb2dedc30e494ebd22c33594cee661. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:08.403829 1932 consensus_queue.cc:1048] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:08.405987 1176 ts_tablet_manager.cc:1434] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.042s user 0.003s sys 0.000s
I20260505 07:54:08.407574 2276 ts_tablet_manager.cc:1916] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:08.408751 1932 consensus_queue.cc:1048] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:08.411473 1176 tablet_bootstrap.cc:492] T 036d306d592f41cb96814aa60e30551e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:54:08.415349 718 leader_election.cc:343] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8eb7eaff8837485a8ae3ccb44c60471d
I20260505 07:54:08.417273 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5fa390b240d4079b8ada7c20adc33a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.417018 718 leader_election.cc:304] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
W20260505 07:54:08.422261 718 raft_consensus.cc:2686] T 8eb7eaff8837485a8ae3ccb44c60471d P eac7e1e8caba43cdb4df80cbb80e891c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260505 07:54:08.424172 2137 ts_tablet_manager.cc:1929] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:08.424350 2137 log.cc:1199] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/e278b86e7b5d444ebad8d0f6921d4c62
I20260505 07:54:08.424813 2137 ts_tablet_manager.cc:1950] T e278b86e7b5d444ebad8d0f6921d4c62 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:08.425853 1181 tablet_bootstrap.cc:492] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:08.425871 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet e278b86e7b5d444ebad8d0f6921d4c62 (table default.Table_633da5b39da34f5c88163c9936fd0212 [id=d53c653124364ea5acc357977c9db864]) successfully deleted
I20260505 07:54:08.426148 2268 tablet_replica.cc:333] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
W20260505 07:54:08.426198 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 25072055f5534078adf5b5025614f951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25072055f5534078adf5b5025614f951 already in progress: creating tablet
I20260505 07:54:08.426331 2268 raft_consensus.cc:2243] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
W20260505 07:54:08.428563 854 leader_election.cc:343] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.428709 854 leader_election.cc:304] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:08.431252 1909 raft_consensus.cc:2749] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:08.431413 854 leader_election.cc:343] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.439366 2268 raft_consensus.cc:2272] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:08.444808 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b406c4c6ba741bc85eef54c6f576459" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:08.451970 2105 consensus_queue.cc:1048] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:08.452359 2268 ts_tablet_manager.cc:1916] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:08.454023 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e227bcd23048402cb0412ef107b766a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.455035 746 tablet_service.cc:1511] Processing CreateTablet for tablet 148a5b1507b24dae9b8215cfb6c5ae9e (DEFAULT_TABLE table=default.Table_f3446fea50fe4259aba56af3f6723b9c [id=edc74053bb4d4cd494578931cd15a18c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.455297 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11bdeb785eb34110be29f54fb835f132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.455425 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 148a5b1507b24dae9b8215cfb6c5ae9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.455847 752 tablet_service.cc:1511] Processing CreateTablet for tablet d7cf5116d3c94209b21f8923e341e756 (DEFAULT_TABLE table=default.Table_da0ef200a5e645e0bef998207f1edcc7 [id=396ca6fffd454cd0a94b9df60166dfaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.456142 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7cf5116d3c94209b21f8923e341e756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.456398 756 tablet_service.cc:1511] Processing CreateTablet for tablet 5d4dfffc0cae451dbc1b1c7622f74047 (DEFAULT_TABLE table=default.Table_8c181aa800d6451dbbc8db3e409c90ae [id=885cac84c8cb46f598e1e64552b06cc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.456666 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d4dfffc0cae451dbc1b1c7622f74047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.463469 899 tablet_service.cc:1511] Processing CreateTablet for tablet 148a5b1507b24dae9b8215cfb6c5ae9e (DEFAULT_TABLE table=default.Table_f3446fea50fe4259aba56af3f6723b9c [id=edc74053bb4d4cd494578931cd15a18c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.463905 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 148a5b1507b24dae9b8215cfb6c5ae9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.464085 2105 raft_consensus.cc:493] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:08.464800 888 tablet_service.cc:1511] Processing CreateTablet for tablet 5d4dfffc0cae451dbc1b1c7622f74047 (DEFAULT_TABLE table=default.Table_8c181aa800d6451dbbc8db3e409c90ae [id=885cac84c8cb46f598e1e64552b06cc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.465185 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d4dfffc0cae451dbc1b1c7622f74047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.466794 759 tablet_service.cc:1511] Processing CreateTablet for tablet a226070ba7db437b83fc0e24ffc0e3e9 (DEFAULT_TABLE table=default.Table_aa70b0997b464af5920413788a8cc6de [id=b024e1563bff4676bbb72bd3f733a975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.467185 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a226070ba7db437b83fc0e24ffc0e3e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.474443 887 tablet_service.cc:1511] Processing CreateTablet for tablet c12b5a36765d4698b64983ab33d23ef8 (DEFAULT_TABLE table=default.Table_c55e50477e284efca5d22c2b75683904 [id=18eba8d2467e4e59a721334135fdf5a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.474910 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c12b5a36765d4698b64983ab33d23ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.475669 756 tablet_service.cc:1511] Processing CreateTablet for tablet c12b5a36765d4698b64983ab33d23ef8 (DEFAULT_TABLE table=default.Table_c55e50477e284efca5d22c2b75683904 [id=18eba8d2467e4e59a721334135fdf5a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.476096 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c12b5a36765d4698b64983ab33d23ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.479453 1172 tablet_bootstrap.cc:654] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.464193 2105 raft_consensus.cc:515] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:08.474515 894 tablet_service.cc:1511] Processing CreateTablet for tablet d7cf5116d3c94209b21f8923e341e756 (DEFAULT_TABLE table=default.Table_da0ef200a5e645e0bef998207f1edcc7 [id=396ca6fffd454cd0a94b9df60166dfaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.483899 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7cf5116d3c94209b21f8923e341e756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.499214 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13800db6031045babf951adbee263377. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.500252 759 tablet_service.cc:1511] Processing CreateTablet for tablet 9926e569032144ec9d3148f4561333ab (DEFAULT_TABLE table=default.Table_f56ca420b05b4508a41ee72179b9c93f [id=ed71bc1104b5423caedd9268977cc018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.500494 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0fea885db74bf4be31d464e90e2292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.504037 746 tablet_service.cc:1511] Processing CreateTablet for tablet 8cf3c334ebb24956b50dbefc23d4d60b (DEFAULT_TABLE table=default.Table_e137326bc9684662bffdc5c59e122c6a [id=199b3d61d922405b9e2a018c1c8c6080]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:08.505769 854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 74131.2us
I20260505 07:54:08.505792 888 tablet_service.cc:1511] Processing CreateTablet for tablet a226070ba7db437b83fc0e24ffc0e3e9 (DEFAULT_TABLE table=default.Table_aa70b0997b464af5920413788a8cc6de [id=b024e1563bff4676bbb72bd3f733a975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.506855 1172 tablet_bootstrap.cc:492] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.506739 2276 ts_tablet_manager.cc:1929] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:08.507139 1172 ts_tablet_manager.cc:1403] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.137s user 0.007s sys 0.007s
I20260505 07:54:08.507164 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a226070ba7db437b83fc0e24ffc0e3e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.507282 2276 log.cc:1199] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/54e4f7f895d943639653c8c32724baf2
I20260505 07:54:08.507974 1172 raft_consensus.cc:359] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.508152 1172 raft_consensus.cc:385] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.508212 1172 raft_consensus.cc:740] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.508399 1172 consensus_queue.cc:260] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.508688 2276 ts_tablet_manager.cc:1950] T 54e4f7f895d943639653c8c32724baf2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:08.508905 1172 ts_tablet_manager.cc:1434] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:08.509166 1172 tablet_bootstrap.cc:492] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.509183 1019 tablet_service.cc:1511] Processing CreateTablet for tablet d7cf5116d3c94209b21f8923e341e756 (DEFAULT_TABLE table=default.Table_da0ef200a5e645e0bef998207f1edcc7 [id=396ca6fffd454cd0a94b9df60166dfaf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.509562 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7cf5116d3c94209b21f8923e341e756. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.510385 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 54e4f7f895d943639653c8c32724baf2 (table default.Table_ceaf4d54d57542928716e3056eac2bde [id=54621fbfaf6a4b4a874207bd986b77be]) successfully deleted
I20260505 07:54:08.510845 899 tablet_service.cc:1511] Processing CreateTablet for tablet 9926e569032144ec9d3148f4561333ab (DEFAULT_TABLE table=default.Table_f56ca420b05b4508a41ee72179b9c93f [id=ed71bc1104b5423caedd9268977cc018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.511242 2276 tablet_replica.cc:333] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:08.511786 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9926e569032144ec9d3148f4561333ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.512733 2276 raft_consensus.cc:2243] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:08.517974 2105 leader_election.cc:290] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:08.507639 1018 tablet_service.cc:1511] Processing CreateTablet for tablet 5d4dfffc0cae451dbc1b1c7622f74047 (DEFAULT_TABLE table=default.Table_8c181aa800d6451dbbc8db3e409c90ae [id=885cac84c8cb46f598e1e64552b06cc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.519006 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d4dfffc0cae451dbc1b1c7622f74047. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.521260 1172 tablet_bootstrap.cc:654] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.524293 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 148a5b1507b24dae9b8215cfb6c5ae9e (DEFAULT_TABLE table=default.Table_f3446fea50fe4259aba56af3f6723b9c [id=edc74053bb4d4cd494578931cd15a18c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.524689 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 148a5b1507b24dae9b8215cfb6c5ae9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.525467 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cf3c334ebb24956b50dbefc23d4d60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.530076 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:08.531061 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:08.531487 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9926e569032144ec9d3148f4561333ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.531759 2308 raft_consensus.cc:493] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:08.531855 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:08.531881 2308 raft_consensus.cc:515] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.532021 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:08.532415 2308 leader_election.cc:290] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:08.534727 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:08.535200 2276 raft_consensus.cc:2272] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:08.535408 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:08.536149 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:08.536207 2276 ts_tablet_manager.cc:1916] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:08.536928 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:08.537292 1033 tablet_service.cc:1511] Processing CreateTablet for tablet a226070ba7db437b83fc0e24ffc0e3e9 (DEFAULT_TABLE table=default.Table_aa70b0997b464af5920413788a8cc6de [id=b024e1563bff4676bbb72bd3f733a975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.537586 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:08.537747 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:08.537964 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:08.538177 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet df3a49b59ed6485f970074391f932482 (table default.Table_7af623f581af498b86c8c3b1adbc6b80 [id=306149ae47a745edabf863d5004d5012]). Expected version 1 got 0
I20260505 07:54:08.538622 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:08.539062 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
I20260505 07:54:08.539479 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:08.541915 1358 catalog_manager.cc:5671] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:08.542973 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_728dc87e78954a7fbbf495d74705e382 [id=2c06ff8eed31432e87f08e9b59049652]). Expected version 2 got 0
I20260505 07:54:08.543625 1358 catalog_manager.cc:5671] T d37dc6df9da5401384a6fc5de868b41c P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:08.543800 756 tablet_service.cc:1511] Processing CreateTablet for tablet 52e2db0240e04d6d8e948ebaedcc7d24 (DEFAULT_TABLE table=default.Table_f93a81c5dfe740819f0f6ea5bf7f1067 [id=a8a6d42a6b6748d590ef5949a9d1e650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.544047 1016 tablet_service.cc:1511] Processing CreateTablet for tablet 9926e569032144ec9d3148f4561333ab (DEFAULT_TABLE table=default.Table_f56ca420b05b4508a41ee72179b9c93f [id=ed71bc1104b5423caedd9268977cc018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.544745 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 42360f8ee4ff47c2af8d287f792207de (table default.Table_829cc346ac834637ab2e2198944a8550 [id=2c91868ddba84103b9a5c78201842a9e]). Expected version 1 got 0
I20260505 07:54:08.545444 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_95fae21889c84b61ad67610d209031b2 [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 1 got 0
I20260505 07:54:08.545495 900 tablet_service.cc:1511] Processing CreateTablet for tablet 52e2db0240e04d6d8e948ebaedcc7d24 (DEFAULT_TABLE table=default.Table_f93a81c5dfe740819f0f6ea5bf7f1067 [id=a8a6d42a6b6748d590ef5949a9d1e650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.545960 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52e2db0240e04d6d8e948ebaedcc7d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.546125 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 08c6ed755fc64080a2310fe9a4e0a68b (table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b]). Expected version 3 got 0
I20260505 07:54:08.546903 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:08.547659 1016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9926e569032144ec9d3148f4561333ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.548027 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:08.548489 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
I20260505 07:54:08.548728 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:08.548776 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52e2db0240e04d6d8e948ebaedcc7d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.548766 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a226070ba7db437b83fc0e24ffc0e3e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.549100 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:08.549376 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:08.552125 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:08.553232 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:08.553426 2268 ts_tablet_manager.cc:1929] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:08.553574 2268 log.cc:1199] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/8b57c1c618484d5d86a846fb982a2240
I20260505 07:54:08.554452 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:08.555222 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:08.555456 759 tablet_service.cc:1511] Processing CreateTablet for tablet 3766ff57fcd5402d98e005a2966568d5 (DEFAULT_TABLE table=default.Table_2634c105c2c041959e881249dddecdaf [id=270d8349c86f42ac9e23fb60513c71cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.555864 759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3766ff57fcd5402d98e005a2966568d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.555823 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:08.556262 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:08.556756 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:08.557217 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]). Expected version 2 got 0
I20260505 07:54:08.558153 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:08.559082 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]). Expected version 2 got 0
I20260505 07:54:08.559968 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet c822fafe72544ca99fde910c8e764755 (table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f]). Expected version 1 got 0
W20260505 07:54:08.560236 856 leader_election.cc:343] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.560407 856 leader_election.cc:304] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:08.560603 856 raft_consensus.cc:2686] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260505 07:54:08.560875 1358 catalog_manager.cc:5671] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:08.561389 746 tablet_service.cc:1511] Processing CreateTablet for tablet 762b3bca39b441db98c96471872325d7 (DEFAULT_TABLE table=default.Table_106ccea898e84018bd7f4c26b5960922 [id=a8311518aa964bfc9236f7664d3bf4b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.560235 894 tablet_service.cc:1511] Processing CreateTablet for tablet 8cf3c334ebb24956b50dbefc23d4d60b (DEFAULT_TABLE table=default.Table_e137326bc9684662bffdc5c59e122c6a [id=199b3d61d922405b9e2a018c1c8c6080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.561712 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 762b3bca39b441db98c96471872325d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.561772 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cf3c334ebb24956b50dbefc23d4d60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.562264 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]). Expected version 2 got 0
I20260505 07:54:08.562708 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:08.563836 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:08.564287 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:08.564807 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:08.565402 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:08.566102 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]). Expected version 1 got 0
I20260505 07:54:08.561303 2308 raft_consensus.cc:493] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:08.569005 1024 tablet_service.cc:1511] Processing CreateTablet for tablet c12b5a36765d4698b64983ab33d23ef8 (DEFAULT_TABLE table=default.Table_c55e50477e284efca5d22c2b75683904 [id=18eba8d2467e4e59a721334135fdf5a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.569582 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:08.571882 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a13ca612f241fc97ea12124677a0b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:08.572072 916 raft_consensus.cc:2468] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:08.572554 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 8cf3c334ebb24956b50dbefc23d4d60b (DEFAULT_TABLE table=default.Table_e137326bc9684662bffdc5c59e122c6a [id=199b3d61d922405b9e2a018c1c8c6080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.576229 2105 raft_consensus.cc:493] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:08.576339 2105 raft_consensus.cc:515] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.579317 888 tablet_service.cc:1511] Processing CreateTablet for tablet 3766ff57fcd5402d98e005a2966568d5 (DEFAULT_TABLE table=default.Table_2634c105c2c041959e881249dddecdaf [id=270d8349c86f42ac9e23fb60513c71cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.579926 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3766ff57fcd5402d98e005a2966568d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.579882 2308 raft_consensus.cc:515] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.580791 2268 ts_tablet_manager.cc:1950] T 8b57c1c618484d5d86a846fb982a2240 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:08.584482 2276 ts_tablet_manager.cc:1929] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:08.585884 2276 log.cc:1199] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/8b406c4c6ba741bc85eef54c6f576459
I20260505 07:54:08.587870 1358 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:08.588299 1172 tablet_bootstrap.cc:492] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.588435 1172 ts_tablet_manager.cc:1403] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.079s user 0.012s sys 0.011s
I20260505 07:54:08.590095 1172 raft_consensus.cc:359] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.590444 1172 raft_consensus.cc:385] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.590554 1172 raft_consensus.cc:740] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.590971 1172 consensus_queue.cc:260] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.592187 1172 ts_tablet_manager.cc:1434] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260505 07:54:08.592825 1172 tablet_bootstrap.cc:492] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.600061 746 tablet_service.cc:1511] Processing CreateTablet for tablet 979af5f6dbdd411197a7ea0fe7485cef (DEFAULT_TABLE table=default.Table_90d28c121c4c41568b9a2eb7f98e2294 [id=92988c7edb4d4205a23b3fd1c6b8d743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.600414 746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979af5f6dbdd411197a7ea0fe7485cef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.607348 752 tablet_service.cc:1511] Processing CreateTablet for tablet 880e326cbea74233a0ec5abb31f5b0d7 (DEFAULT_TABLE table=default.Table_865701615b5f4c06884f469ea75d36f8 [id=c618f86fa01f4d4786eb34de77cdd560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.607697 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 880e326cbea74233a0ec5abb31f5b0d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.608155 885 tablet_service.cc:1511] Processing CreateTablet for tablet 979af5f6dbdd411197a7ea0fe7485cef (DEFAULT_TABLE table=default.Table_90d28c121c4c41568b9a2eb7f98e2294 [id=92988c7edb4d4205a23b3fd1c6b8d743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.608517 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979af5f6dbdd411197a7ea0fe7485cef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.611835 892 tablet_service.cc:1511] Processing CreateTablet for tablet 880e326cbea74233a0ec5abb31f5b0d7 (DEFAULT_TABLE table=default.Table_865701615b5f4c06884f469ea75d36f8 [id=c618f86fa01f4d4786eb34de77cdd560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.611986 2276 ts_tablet_manager.cc:1950] T 8b406c4c6ba741bc85eef54c6f576459 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:08.612141 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 880e326cbea74233a0ec5abb31f5b0d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.612267 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 60) took 1019 ms. Trace:
I20260505 07:54:08.612334 2276 rpcz_store.cc:276] 0505 07:54:07.592463 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.592511 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:08.510913 (+918402us) ts_tablet_manager.cc:1151] Deleting tablet 8b406c4c6ba741bc85eef54c6f576459
0505 07:54:08.511089 (+ 176us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:08.541489 (+ 30400us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.584158 (+ 42669us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.612253 (+ 28095us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21863,"tablet-delete.queue_time_us":917487}
I20260505 07:54:08.612767 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 8b406c4c6ba741bc85eef54c6f576459 (table default.Table_d4d6e315fb8340e9b629d876d050ef05 [id=5fdbc0ad9c3f4e0a9b2fc0997a87870b]) successfully deleted
I20260505 07:54:08.613605 2276 tablet_replica.cc:333] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:08.614518 2276 raft_consensus.cc:2243] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:08.614565 2284 raft_consensus.cc:493] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:08.615104 2276 raft_consensus.cc:2272] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:08.616101 2276 ts_tablet_manager.cc:1916] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:08.617810 887 tablet_service.cc:1511] Processing CreateTablet for tablet 762b3bca39b441db98c96471872325d7 (DEFAULT_TABLE table=default.Table_106ccea898e84018bd7f4c26b5960922 [id=a8311518aa964bfc9236f7664d3bf4b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.619011 752 tablet_service.cc:1511] Processing CreateTablet for tablet f57e14ee70584d258c83ff1b01189713 (DEFAULT_TABLE table=default.Table_7ed74bd3c32f49ef8fbdeb8b065b2913 [id=211078866c5443b693140721c13e8827]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.619382 752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f57e14ee70584d258c83ff1b01189713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.622161 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 762b3bca39b441db98c96471872325d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.614892 2284 raft_consensus.cc:515] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.624037 2284 leader_election.cc:290] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:08.624558 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f200abef13ac42698a9b96fc8af82ff0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:08.625144 854 leader_election.cc:343] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.628290 894 tablet_service.cc:1511] Processing CreateTablet for tablet f57e14ee70584d258c83ff1b01189713 (DEFAULT_TABLE table=default.Table_7ed74bd3c32f49ef8fbdeb8b065b2913 [id=211078866c5443b693140721c13e8827]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.628736 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f57e14ee70584d258c83ff1b01189713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.629087 2105 leader_election.cc:290] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:08.638374 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c12b5a36765d4698b64983ab33d23ef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.640080 1176 tablet_bootstrap.cc:654] T 036d306d592f41cb96814aa60e30551e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:08.648608 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 233425us
I20260505 07:54:08.649221 1932 consensus_queue.cc:1048] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:08.653225 2368 consensus_queue.cc:1048] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:08.654366 718 leader_election.cc:304] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:08.654896 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f200abef13ac42698a9b96fc8af82ff0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:08.655555 856 leader_election.cc:343] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.655697 856 leader_election.cc:304] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:08.656144 2284 raft_consensus.cc:2749] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:08.656729 991 leader_election.cc:343] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 840eebf9e7404bf7807b7e51996957e5
W20260505 07:54:08.656853 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 79299us
I20260505 07:54:08.657064 991 leader_election.cc:304] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:08.657148 2308 leader_election.cc:290] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:08.657352 2308 raft_consensus.cc:2804] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:08.657413 2308 raft_consensus.cc:493] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:08.657481 2308 raft_consensus.cc:3060] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:08.659152 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 8) took 1815 ms. Trace:
I20260505 07:54:08.659237 2268 rpcz_store.cc:276] 0505 07:54:06.843689 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.892339 (+ 48650us) service_pool.cc:224] Handling call
0505 07:54:08.270660 (+1378321us) ts_tablet_manager.cc:1151] Deleting tablet 8b57c1c618484d5d86a846fb982a2240
0505 07:54:08.426106 (+155446us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:08.530250 (+104144us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.553383 (+ 23133us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.659136 (+105753us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1378114}
I20260505 07:54:08.666895 1172 tablet_bootstrap.cc:654] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.668085 2268 tablet_replica.cc:333] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:08.669562 2268 raft_consensus.cc:2243] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:08.675706 1878 raft_consensus.cc:493] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:08.675877 1878 raft_consensus.cc:515] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.676534 1878 leader_election.cc:290] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:08.684283 1172 tablet_bootstrap.cc:492] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.684444 1172 ts_tablet_manager.cc:1403] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.092s user 0.002s sys 0.002s
I20260505 07:54:08.685261 1172 raft_consensus.cc:359] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.685421 1172 raft_consensus.cc:385] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.685485 1172 raft_consensus.cc:740] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.685701 1172 consensus_queue.cc:260] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.686194 1172 ts_tablet_manager.cc:1434] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:08.686510 1172 tablet_bootstrap.cc:492] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.672322 2105 consensus_queue.cc:1048] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260505 07:54:08.689494 1172 tablet_bootstrap.cc:654] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.690713 2268 raft_consensus.cc:2272] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:08.692719 2268 ts_tablet_manager.cc:1916] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:08.700528 1172 tablet_bootstrap.cc:492] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.700654 1172 ts_tablet_manager.cc:1403] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.008s
I20260505 07:54:08.701306 1172 raft_consensus.cc:359] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.701447 1172 raft_consensus.cc:385] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.701495 1172 raft_consensus.cc:740] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.701691 1172 consensus_queue.cc:260] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.702297 1172 ts_tablet_manager.cc:1434] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:08.702601 1172 tablet_bootstrap.cc:492] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.704304 1172 tablet_bootstrap.cc:654] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.718549 1172 tablet_bootstrap.cc:492] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.718703 1172 ts_tablet_manager.cc:1403] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.008s
I20260505 07:54:08.719635 1172 raft_consensus.cc:359] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.719813 1172 raft_consensus.cc:385] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.719864 1172 raft_consensus.cc:740] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.720429 1172 consensus_queue.cc:260] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.721374 1172 ts_tablet_manager.cc:1434] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:08.721635 1172 tablet_bootstrap.cc:492] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.725442 1172 tablet_bootstrap.cc:654] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.726158 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8cf3c334ebb24956b50dbefc23d4d60b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.734948 1172 tablet_bootstrap.cc:492] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.735073 1172 ts_tablet_manager.cc:1403] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.001s
I20260505 07:54:08.737458 1172 raft_consensus.cc:359] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.737613 1172 raft_consensus.cc:385] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.737664 1172 raft_consensus.cc:740] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.737836 1172 consensus_queue.cc:260] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.741348 1172 ts_tablet_manager.cc:1434] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260505 07:54:08.750077 2392 raft_consensus.cc:493] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:08.750190 2392 raft_consensus.cc:515] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.750924 2392 leader_election.cc:290] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:08.751716 1021 tablet_service.cc:1511] Processing CreateTablet for tablet 3766ff57fcd5402d98e005a2966568d5 (DEFAULT_TABLE table=default.Table_2634c105c2c041959e881249dddecdaf [id=270d8349c86f42ac9e23fb60513c71cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.751858 1172 tablet_bootstrap.cc:492] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.751787 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76939b7978ee4f7ea0ad29312de2a31a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.752091 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3766ff57fcd5402d98e005a2966568d5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:08.752243 854 leader_election.cc:343] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.753743 1172 tablet_bootstrap.cc:654] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.762375 1172 tablet_bootstrap.cc:492] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.762504 1172 ts_tablet_manager.cc:1403] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.001s
I20260505 07:54:08.763176 1172 raft_consensus.cc:359] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.768196 1172 raft_consensus.cc:385] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.768272 1172 raft_consensus.cc:740] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.768544 1172 consensus_queue.cc:260] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.769452 1172 ts_tablet_manager.cc:1434] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:54:08.770593 1172 tablet_bootstrap.cc:492] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.777659 1172 tablet_bootstrap.cc:654] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.783411 1172 tablet_bootstrap.cc:492] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.783543 1172 ts_tablet_manager.cc:1403] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260505 07:54:08.785084 2308 raft_consensus.cc:515] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.785262 1172 raft_consensus.cc:359] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.785387 1172 raft_consensus.cc:385] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.785436 1172 raft_consensus.cc:740] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.786248 1176 tablet_bootstrap.cc:492] T 036d306d592f41cb96814aa60e30551e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:08.786369 1176 ts_tablet_manager.cc:1403] T 036d306d592f41cb96814aa60e30551e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.376s user 0.003s sys 0.000s
I20260505 07:54:08.787290 1172 consensus_queue.cc:260] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.791303 1172 ts_tablet_manager.cc:1434] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260505 07:54:08.791692 1172 tablet_bootstrap.cc:492] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.793838 1172 tablet_bootstrap.cc:654] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.802775 1172 tablet_bootstrap.cc:492] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.802888 1172 ts_tablet_manager.cc:1403] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.002s
I20260505 07:54:08.804100 1172 raft_consensus.cc:359] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.805500 1172 raft_consensus.cc:385] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.805559 1172 raft_consensus.cc:740] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.805783 1172 consensus_queue.cc:260] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.806221 1172 ts_tablet_manager.cc:1434] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:08.806459 1172 tablet_bootstrap.cc:492] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.808073 1172 tablet_bootstrap.cc:654] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.809064 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 762b3bca39b441db98c96471872325d7 (DEFAULT_TABLE table=default.Table_106ccea898e84018bd7f4c26b5960922 [id=a8311518aa964bfc9236f7664d3bf4b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.809749 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 762b3bca39b441db98c96471872325d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.813969 2276 ts_tablet_manager.cc:1929] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:08.815932 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:08.816355 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:08.816673 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:08.817096 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:08.817550 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:08.817723 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:08.817950 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:08.818282 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:08.818473 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet df3a49b59ed6485f970074391f932482 (table default.Table_7af623f581af498b86c8c3b1adbc6b80 [id=306149ae47a745edabf863d5004d5012]). Expected version 1 got 0
I20260505 07:54:08.818842 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:08.819087 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:08.819340 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:08.821226 2368 raft_consensus.cc:2749] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:08.822257 1176 raft_consensus.cc:359] T 036d306d592f41cb96814aa60e30551e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:08.822415 1176 raft_consensus.cc:385] T 036d306d592f41cb96814aa60e30551e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.822456 1176 raft_consensus.cc:740] T 036d306d592f41cb96814aa60e30551e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.822621 1176 consensus_queue.cc:260] T 036d306d592f41cb96814aa60e30551e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:08.823201 1176 ts_tablet_manager.cc:1434] T 036d306d592f41cb96814aa60e30551e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.037s user 0.001s sys 0.000s
I20260505 07:54:08.823551 1176 tablet_bootstrap.cc:492] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:08.825022 1176 tablet_bootstrap.cc:654] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.819514 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
W20260505 07:54:08.842533 1322 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5006965919824adbbb23d744fb9ff335 consensus state has no leader
I20260505 07:54:08.842258 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:08.845382 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:08.856303 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 52e2db0240e04d6d8e948ebaedcc7d24 (DEFAULT_TABLE table=default.Table_f93a81c5dfe740819f0f6ea5bf7f1067 [id=a8a6d42a6b6748d590ef5949a9d1e650]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.862501 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52e2db0240e04d6d8e948ebaedcc7d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.863426 2308 leader_election.cc:290] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:08.868014 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 08c6ed755fc64080a2310fe9a4e0a68b (table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b]). Expected version 3 got 0
W20260505 07:54:08.868184 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 213826us
I20260505 07:54:08.868574 718 leader_election.cc:304] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:08.869933 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_95fae21889c84b61ad67610d209031b2 [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 1 got 0
I20260505 07:54:08.870106 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 979af5f6dbdd411197a7ea0fe7485cef (DEFAULT_TABLE table=default.Table_90d28c121c4c41568b9a2eb7f98e2294 [id=92988c7edb4d4205a23b3fd1c6b8d743]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.871734 1181 tablet_bootstrap.cc:654] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.878758 2276 log.cc:1199] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/43e4c92b7c3140fdb2d6a290aaf0ad55
I20260505 07:54:08.879323 2276 ts_tablet_manager.cc:1950] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:08.880254 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 59) took 1288 ms. Trace:
I20260505 07:54:08.880401 2276 rpcz_store.cc:276] 0505 07:54:07.591748 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.591818 (+ 70us) service_pool.cc:224] Handling call
0505 07:54:08.613470 (+1021652us) ts_tablet_manager.cc:1151] Deleting tablet 43e4c92b7c3140fdb2d6a290aaf0ad55
0505 07:54:08.613570 (+ 100us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:08.682831 (+ 69261us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.813918 (+131087us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.880234 (+ 66316us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1019873}
I20260505 07:54:08.880941 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 (table default.Table_43b383b7a58b493c8f5b2d9f3038a2e2 [id=5154509860404937ba84ef134bb50cef]) successfully deleted
I20260505 07:54:08.881572 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 57) took 1292 ms. Trace:
I20260505 07:54:08.884889 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:08.885768 1019 tablet_service.cc:1511] Processing CreateTablet for tablet 880e326cbea74233a0ec5abb31f5b0d7 (DEFAULT_TABLE table=default.Table_865701615b5f4c06884f469ea75d36f8 [id=c618f86fa01f4d4786eb34de77cdd560]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.885995 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:08.886003 2276 rpcz_store.cc:276] 0505 07:54:07.589090 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.594740 (+ 5650us) service_pool.cc:224] Handling call
0505 07:54:08.881431 (+1286691us) ts_tablet_manager.cc:1151] Deleting tablet a9b7c6e9463342a5a73af12da4447f89
0505 07:54:08.881443 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:08.881554 (+ 111us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1286451}
W20260505 07:54:08.886448 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a9b7c6e9463342a5a73af12da4447f89 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9b7c6e9463342a5a73af12da4447f89 already in progress: creating tablet
I20260505 07:54:08.886780 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 58) took 1295 ms. Trace:
I20260505 07:54:08.886864 2276 rpcz_store.cc:276] 0505 07:54:07.591654 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.595028 (+ 3374us) service_pool.cc:224] Handling call
0505 07:54:08.886678 (+1291650us) ts_tablet_manager.cc:1151] Deleting tablet e82e9ae417784b7e83f4843b129d0580
0505 07:54:08.886686 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:08.886765 (+ 79us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1268677}
W20260505 07:54:08.887359 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e82e9ae417784b7e83f4843b129d0580 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e82e9ae417784b7e83f4843b129d0580 already in progress: creating tablet
I20260505 07:54:08.887666 2276 tablet_replica.cc:333] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:08.887903 2276 raft_consensus.cc:2243] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:08.888049 2276 raft_consensus.cc:2272] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:08.890236 718 consensus_peers.cc:597] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:08.899876 2412 raft_consensus.cc:2749] T 934cea6666b1420eb12fe4510d2466fe P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:08.902552 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 80754.4us
I20260505 07:54:08.925114 1176 tablet_bootstrap.cc:492] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:08.925282 1176 ts_tablet_manager.cc:1403] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.102s user 0.003s sys 0.019s
I20260505 07:54:08.926306 1176 raft_consensus.cc:359] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.926558 1176 raft_consensus.cc:385] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.926653 1176 raft_consensus.cc:740] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.926858 1176 consensus_queue.cc:260] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.927392 1176 ts_tablet_manager.cc:1434] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:08.927806 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a13ca612f241fc97ea12124677a0b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.932524 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727cacd242844455915bd9963d2db19f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.933106 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4e5af08ddc4c77b5640b8295b236b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.933607 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca12a17fb014374bf7ed764a91e6985" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.934926 1176 tablet_bootstrap.cc:492] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:08.888548 2276 ts_tablet_manager.cc:1916] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:08.930800 1172 tablet_bootstrap.cc:492] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:08.936895 1172 ts_tablet_manager.cc:1403] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.131s user 0.010s sys 0.018s
I20260505 07:54:08.937636 1172 raft_consensus.cc:359] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.937815 1172 raft_consensus.cc:385] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:08.937883 1172 raft_consensus.cc:740] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:08.938074 2414 consensus_queue.cc:1048] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:08.938099 1172 consensus_queue.cc:260] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.938607 1172 ts_tablet_manager.cc:1434] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:08.938880 1172 tablet_bootstrap.cc:492] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:08.939437 2433 tablet.cc:1724] T c822fafe72544ca99fde910c8e764755 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:08.940331 1172 tablet_bootstrap.cc:654] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.943444 1176 tablet_bootstrap.cc:654] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:08.958364 2284 raft_consensus.cc:493] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:08.958496 2284 raft_consensus.cc:515] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:08.959040 2284 leader_election.cc:290] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:08.959928 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:08.959946 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66082a7a75e345639e7a268727922fbb" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:08.960318 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
W20260505 07:54:08.960433 854 leader_election.cc:343] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:08.960680 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 42360f8ee4ff47c2af8d287f792207de (table default.Table_829cc346ac834637ab2e2198944a8550 [id=2c91868ddba84103b9a5c78201842a9e]). Expected version 1 got 0
I20260505 07:54:08.961133 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:08.961388 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:08.961848 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:08.963383 1181 tablet_bootstrap.cc:492] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:08.968271 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 880e326cbea74233a0ec5abb31f5b0d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.968619 1181 ts_tablet_manager.cc:1403] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.543s user 0.000s sys 0.009s
I20260505 07:54:08.969871 2414 consensus_queue.cc:1048] T 5191f9a357de4607aab6c7a8b4bcccdb P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:08.970002 1035 consensus_meta.cc:287] T 38eee32a6fd84f89a1e88074f16d4242 P d63a3e535f8e4dd4a88e447253405b77: Time spent flushing consensus metadata: real 0.522s user 0.004s sys 0.000s
I20260505 07:54:08.970382 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]). Expected version 2 got 0
I20260505 07:54:08.970702 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:08.970969 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:08.971100 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet c822fafe72544ca99fde910c8e764755 (table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f]). Expected version 1 got 0
I20260505 07:54:08.971303 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:08.971438 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
I20260505 07:54:08.971724 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]). Expected version 1 got 0
I20260505 07:54:08.972393 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 8d1ce00e487d4702a6affcad57a7fbe1 (table default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b]). Expected version 2 got 0
I20260505 07:54:08.972707 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]). Expected version 2 got 0
I20260505 07:54:08.973081 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:08.973369 1020 tablet_service.cc:1511] Processing CreateTablet for tablet f57e14ee70584d258c83ff1b01189713 (DEFAULT_TABLE table=default.Table_7ed74bd3c32f49ef8fbdeb8b065b2913 [id=211078866c5443b693140721c13e8827]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:08.973603 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:08.973757 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f57e14ee70584d258c83ff1b01189713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.973912 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:08.974823 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 8b57c1c618484d5d86a846fb982a2240 (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]) successfully deleted
I20260505 07:54:08.981201 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4e5af08ddc4c77b5640b8295b236b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:08.981412 920 raft_consensus.cc:2468] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:08.983997 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca12a17fb014374bf7ed764a91e6985" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:08.984177 913 raft_consensus.cc:2468] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:08.989585 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 979af5f6dbdd411197a7ea0fe7485cef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:08.991138 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:08.992007 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:08.993140 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:08.993712 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a13ca612f241fc97ea12124677a0b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:08.993556 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_728dc87e78954a7fbbf495d74705e382 [id=2c06ff8eed31432e87f08e9b59049652]). Expected version 2 got 0
I20260505 07:54:08.993989 921 raft_consensus.cc:3060] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:08.995489 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:08.994863 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727cacd242844455915bd9963d2db19f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:08.996361 911 raft_consensus.cc:2468] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:08.997735 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
I20260505 07:54:08.999548 2268 ts_tablet_manager.cc:1929] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:08.999704 2268 log.cc:1199] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/bde59d52fd234343b1ca13a7d4aae6bf
I20260505 07:54:09.003000 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:09.006801 1355 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:09.009092 921 raft_consensus.cc:2468] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:09.011708 2268 ts_tablet_manager.cc:1950] T bde59d52fd234343b1ca13a7d4aae6bf P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:09.012008 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 9) took 2119 ms. Trace:
I20260505 07:54:09.012084 2268 rpcz_store.cc:276] 0505 07:54:06.892687 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.892726 (+ 39us) service_pool.cc:224] Handling call
0505 07:54:08.668006 (+1775280us) ts_tablet_manager.cc:1151] Deleting tablet bde59d52fd234343b1ca13a7d4aae6bf
0505 07:54:08.668020 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:08.803659 (+135639us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:08.999500 (+195841us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.011993 (+ 12493us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1775107}
I20260505 07:54:09.012400 2268 tablet_replica.cc:333] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
W20260505 07:54:09.020879 1352 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 21480c42b73e4bf0ab2453b085a867c0 consensus state has no leader
I20260505 07:54:09.021056 1352 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 105) took 1395 ms. Trace:
I20260505 07:54:09.021150 1352 rpcz_store.cc:276] 0505 07:54:07.625939 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.626168 (+ 229us) service_pool.cc:224] Handling call
0505 07:54:07.712987 (+ 86819us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:07.713191 (+ 204us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:07.713325 (+ 134us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:07.713404 (+ 79us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:07.861257 (+147853us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:09.021023 (+1159766us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:07.773541 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:07.773672 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:07.773677 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:07.773680 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:07.773782 (+ 102us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:07.773802 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:07.773807 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:07.773893 (+ 86us) tablet.cc:701] Row locks acquired
0505 07:54:07.773895 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:07.773928 (+ 33us) write_op.cc:270] Start()
0505 07:54:07.773957 (+ 29us) write_op.cc:276] Timestamp: P: 1777967647773897 usec, L: 0
0505 07:54:07.773962 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:07.848372 (+ 74410us) log.cc:844] Serialized 405 byte log entry
0505 07:54:07.848584 (+ 212us) spinlock_profiling.cc:243] Waited 74.2 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65bae8f22 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac
0505 07:54:07.860018 (+ 11434us) op_driver.cc:464] REPLICATION: finished
0505 07:54:07.860196 (+ 178us) write_op.cc:301] APPLY: starting
0505 07:54:07.860215 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:07.860269 (+ 54us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:07.860273 (+ 4us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:07.860334 (+ 61us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:07.860381 (+ 47us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:07.860392 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:07.860478 (+ 86us) log.cc:844] Serialized 23 byte log entry
0505 07:54:07.860556 (+ 78us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.860592 (+ 36us) write_op.cc:454] Released schema lock
0505 07:54:07.860600 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:07.860681 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.860687 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":121,"apply.run_cpu_time_us":473,"apply.run_wall_time_us":473,"num_ops":1,"prepare.queue_time_us":41,"prepare.run_cpu_time_us":1913,"prepare.run_wall_time_us":75131,"raft.queue_time_us":1405,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":340,"replication_time_us":86017,"spinlock_wait_cycles":170611584}]]}
W20260505 07:54:09.042681 1350 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aacf715cb07d4fe9b2c57caea594ab87 consensus state has no leader
I20260505 07:54:09.043774 1348 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 72da22d790624b07bfca6f6b6301c925
I20260505 07:54:09.044266 1348 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 92) took 2072 ms. Trace:
I20260505 07:54:09.044373 1348 rpcz_store.cc:276] 0505 07:54:06.972208 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.420833 (+448625us) service_pool.cc:224] Handling call
0505 07:54:07.421248 (+ 415us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:07.421388 (+ 140us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:07.421474 (+ 86us) catalog_manager.cc:2824] Locking tablets
0505 07:54:07.611422 (+189948us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:07.626379 (+ 14957us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:07.703755 (+ 77376us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:09.043705 (+1339950us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:09.044229 (+ 524us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:07.612224 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:07.612352 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:07.612361 (+ 9us) write_op.cc:435] Acquired schema lock
0505 07:54:07.612365 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:07.612477 (+ 112us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:07.612493 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:07.612497 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:07.612661 (+ 164us) tablet.cc:701] Row locks acquired
0505 07:54:07.612664 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:07.612697 (+ 33us) write_op.cc:270] Start()
0505 07:54:07.612728 (+ 31us) write_op.cc:276] Timestamp: P: 1777967647612637 usec, L: 0
0505 07:54:07.612734 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:07.612901 (+ 167us) log.cc:844] Serialized 756 byte log entry
0505 07:54:07.615221 (+ 2320us) op_driver.cc:464] REPLICATION: finished
0505 07:54:07.616964 (+ 1743us) write_op.cc:301] APPLY: starting
0505 07:54:07.616978 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:07.625682 (+ 8704us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:07.625728 (+ 46us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:07.625810 (+ 82us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:07.625852 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:07.625859 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:07.625933 (+ 74us) log.cc:844] Serialized 29 byte log entry
0505 07:54:07.625993 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.626015 (+ 22us) write_op.cc:454] Released schema lock
0505 07:54:07.626023 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:07.626089 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.626092 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1691,"apply.run_cpu_time_us":476,"apply.run_wall_time_us":9120,"num_ops":2,"prepare.queue_time_us":50,"prepare.run_cpu_time_us":766,"prepare.run_wall_time_us":765,"raft.queue_time_us":61,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":186,"replication_time_us":2461}]]}
I20260505 07:54:09.045214 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_c00d8489b8f84ee3b9eb57d5f51514ba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:09.045782 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c00d8489b8f84ee3b9eb57d5f51514ba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:09.046101 2446 tablet.cc:1724] T 08c6ed755fc64080a2310fe9a4e0a68b P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:09.046725 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:09.051295 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 72da22d790624b07bfca6f6b6301c925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:09.052901 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 72da22d790624b07bfca6f6b6301c925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.058396 2392 raft_consensus.cc:493] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.058531 2392 raft_consensus.cc:515] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.059198 2392 leader_election.cc:290] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.060081 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84f464e83cb412eade9978ea99f6685" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:09.060667 854 leader_election.cc:343] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.067055 1181 raft_consensus.cc:359] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.067245 1181 raft_consensus.cc:385] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.067302 1181 raft_consensus.cc:740] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.067524 1181 consensus_queue.cc:260] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.068025 1181 ts_tablet_manager.cc:1434] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.099s user 0.000s sys 0.002s
I20260505 07:54:09.068220 2445 tablet.cc:1724] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:09.069519 1354 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf
I20260505 07:54:09.070052 1354 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 109) took 1425 ms. Trace:
I20260505 07:54:09.070133 1354 rpcz_store.cc:276] 0505 07:54:07.644064 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.704127 (+ 60063us) service_pool.cc:224] Handling call
0505 07:54:07.784362 (+ 80235us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:07.784583 (+ 221us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:07.784724 (+ 141us) catalog_manager.cc:2824] Locking tablets
0505 07:54:07.784915 (+ 191us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:07.861374 (+ 76459us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:07.861461 (+ 87us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:09.069471 (+1208010us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:09.070001 (+ 530us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:07.848688 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:07.848862 (+ 174us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:07.848869 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:07.848873 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:07.848994 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:07.849006 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:07.849010 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:07.849112 (+ 102us) tablet.cc:701] Row locks acquired
0505 07:54:07.849115 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:07.849161 (+ 46us) write_op.cc:270] Start()
0505 07:54:07.849194 (+ 33us) write_op.cc:276] Timestamp: P: 1777967647849121 usec, L: 0
0505 07:54:07.849197 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:07.849336 (+ 139us) log.cc:844] Serialized 790 byte log entry
0505 07:54:07.860098 (+ 10762us) op_driver.cc:464] REPLICATION: finished
0505 07:54:07.860759 (+ 661us) write_op.cc:301] APPLY: starting
0505 07:54:07.860771 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:07.860810 (+ 39us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:07.860846 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:07.860912 (+ 66us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:07.860948 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:07.860959 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:07.861019 (+ 60us) log.cc:844] Serialized 29 byte log entry
0505 07:54:07.861061 (+ 42us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.861076 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:07.861084 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:07.861148 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.861154 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":626,"apply.run_cpu_time_us":379,"apply.run_wall_time_us":379,"num_ops":2,"prepare.queue_time_us":62879,"prepare.run_cpu_time_us":726,"prepare.run_wall_time_us":726,"raft.queue_time_us":1019,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":10881}]]}
I20260505 07:54:09.070919 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_7ff5390dc98f47729200b3f768305c40"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:09.071463 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7ff5390dc98f47729200b3f768305c40 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:09.072409 887 tablet_service.cc:1558] Processing DeleteTablet for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:09.073935 759 tablet_service.cc:1558] Processing DeleteTablet for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.075811 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet bde59d52fd234343b1ca13a7d4aae6bf (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]) successfully deleted
W20260505 07:54:09.077997 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 76939b7978ee4f7ea0ad29312de2a31a consensus state has no leader
I20260505 07:54:09.086266 1181 tablet_bootstrap.cc:492] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:09.078120 991 leader_election.cc:304] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:09.088102 991 leader_election.cc:304] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:09.088326 2402 raft_consensus.cc:2804] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:09.088392 2402 raft_consensus.cc:493] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:09.088459 2402 raft_consensus.cc:3060] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:09.091006 2402 raft_consensus.cc:515] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.078322 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.078469 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 72da22d790624b07bfca6f6b6301c925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.091974 2308 raft_consensus.cc:2804] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:09.092468 2308 raft_consensus.cc:493] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:09.092548 2308 raft_consensus.cc:3060] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:09.094961 2308 raft_consensus.cc:515] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.095660 2308 leader_election.cc:290] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:09.091572 2402 leader_election.cc:290] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:09.098002 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76939b7978ee4f7ea0ad29312de2a31a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.101962 2268 raft_consensus.cc:2243] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:09.102145 2268 raft_consensus.cc:2272] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.102732 2268 ts_tablet_manager.cc:1916] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:09.110632 2268 ts_tablet_manager.cc:1929] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:09.110889 2268 log.cc:1199] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/840eebf9e7404bf7807b7e51996957e5
I20260505 07:54:09.111451 2268 ts_tablet_manager.cc:1950] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:09.111851 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 10) took 2218 ms. Trace:
W20260505 07:54:09.131490 856 leader_election.cc:343] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.134657 1172 tablet_bootstrap.cc:492] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.135504 856 leader_election.cc:304] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.135622 1172 ts_tablet_manager.cc:1403] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.197s user 0.021s sys 0.037s
I20260505 07:54:09.135967 2444 raft_consensus.cc:2749] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.136447 1172 raft_consensus.cc:359] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.136667 1172 raft_consensus.cc:385] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.136744 1172 raft_consensus.cc:740] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.137002 1172 consensus_queue.cc:260] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.138306 1172 ts_tablet_manager.cc:1434] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:09.138774 1172 tablet_bootstrap.cc:492] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.140609 1172 tablet_bootstrap.cc:654] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.146564 1172 tablet_bootstrap.cc:492] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.146731 1172 ts_tablet_manager.cc:1403] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260505 07:54:09.149538 1172 raft_consensus.cc:359] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.149746 1172 raft_consensus.cc:385] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.149807 1172 raft_consensus.cc:740] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.150054 1172 consensus_queue.cc:260] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.150581 1172 ts_tablet_manager.cc:1434] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260505 07:54:09.150897 1172 tablet_bootstrap.cc:492] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.152837 1172 tablet_bootstrap.cc:654] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.158918 2433 tablet.cc:1724] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:09.092064 991 leader_election.cc:304] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
W20260505 07:54:09.161103 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 69048.4us
I20260505 07:54:09.115356 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84f464e83cb412eade9978ea99f6685" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:09.161375 991 consensus_peers.cc:597] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:09.161571 991 leader_election.cc:304] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:09.124663 2402 raft_consensus.cc:493] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.161890 2402 raft_consensus.cc:515] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:09.162240 2308 raft_consensus.cc:2804] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:09.162380 2307 consensus_queue.cc:1048] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.162446 2402 leader_election.cc:290] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:09.162597 2308 raft_consensus.cc:697] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:09.162846 2308 consensus_queue.cc:237] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.164093 2392 raft_consensus.cc:493] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.164227 2392 raft_consensus.cc:515] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.164883 2392 leader_election.cc:290] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.165894 1172 tablet_bootstrap.cc:492] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.166057 1172 ts_tablet_manager.cc:1403] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260505 07:54:09.166923 1172 raft_consensus.cc:359] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.167124 1172 raft_consensus.cc:385] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.167196 1172 raft_consensus.cc:740] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.169059 2307 consensus_queue.cc:1048] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.169309 1172 consensus_queue.cc:260] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.169911 1172 ts_tablet_manager.cc:1434] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:09.170222 1172 tablet_bootstrap.cc:492] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.170289 2307 consensus_queue.cc:1048] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.171271 2307 consensus_queue.cc:1048] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.173666 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4e5af08ddc4c77b5640b8295b236b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:09.173882 919 raft_consensus.cc:3060] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:09.174404 1172 tablet_bootstrap.cc:654] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.175706 2414 raft_consensus.cc:493] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.175829 2414 raft_consensus.cc:515] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:09.176461 1353 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d] alter complete (version 3)
I20260505 07:54:09.182312 1172 tablet_bootstrap.cc:492] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.182464 1172 ts_tablet_manager.cc:1403] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260505 07:54:09.183343 1172 raft_consensus.cc:359] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.183502 1172 raft_consensus.cc:385] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.183548 1172 raft_consensus.cc:740] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.185694 1172 consensus_queue.cc:260] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.186295 1172 ts_tablet_manager.cc:1434] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:09.186597 1172 tablet_bootstrap.cc:492] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.187768 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca12a17fb014374bf7ed764a91e6985" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:09.187929 912 raft_consensus.cc:3060] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:09.188432 1172 tablet_bootstrap.cc:654] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.189652 919 raft_consensus.cc:2468] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:09.130077 1181 tablet_bootstrap.cc:654] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.175556 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90a13ca612f241fc97ea12124677a0b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:09.211568 2414 leader_election.cc:290] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:09.165488 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1175f7deec14fe39098ec89ab3bc049" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.215812 2450 raft_consensus.cc:493] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 23c57a2c927c4c54bccecfeb3a775311)
I20260505 07:54:09.215983 2450 raft_consensus.cc:515] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:54:09.217132 767 raft_consensus.cc:3060] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:09.219441 2268 rpcz_store.cc:276] 0505 07:54:06.893010 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.893049 (+ 39us) service_pool.cc:224] Handling call
0505 07:54:09.012359 (+2119310us) ts_tablet_manager.cc:1151] Deleting tablet 840eebf9e7404bf7807b7e51996957e5
0505 07:54:09.012367 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.107239 (+ 94872us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.110567 (+ 3328us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.111835 (+ 1268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89334,"tablet-delete.queue_time_us":2119143}
I20260505 07:54:09.220146 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 11) took 2326 ms. Trace:
I20260505 07:54:09.220314 2268 rpcz_store.cc:276] 0505 07:54:06.893312 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.893346 (+ 34us) service_pool.cc:224] Handling call
0505 07:54:09.219988 (+2326642us) ts_tablet_manager.cc:1151] Deleting tablet 02716f62aa354d8f839266661b7f05ea
0505 07:54:09.220002 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.220126 (+ 124us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2326484}
I20260505 07:54:09.220821 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 39) took 2138 ms. Trace:
I20260505 07:54:09.220952 2268 rpcz_store.cc:276] 0505 07:54:07.082430 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.114089 (+ 31659us) service_pool.cc:224] Handling call
0505 07:54:09.220697 (+2106608us) ts_tablet_manager.cc:1151] Deleting tablet abaa596871bb4a82aefc317932e91d35
0505 07:54:09.220709 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.220805 (+ 96us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2106444}
I20260505 07:54:09.221304 2268 tablet_replica.cc:333] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:09.192520 912 raft_consensus.cc:2468] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:09.199486 1172 tablet_bootstrap.cc:492] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.223191 1172 ts_tablet_manager.cc:1403] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.037s user 0.004s sys 0.002s
I20260505 07:54:09.223976 1172 raft_consensus.cc:359] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.224136 1172 raft_consensus.cc:385] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.224192 1172 raft_consensus.cc:740] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.224372 1172 consensus_queue.cc:260] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.224867 1172 ts_tablet_manager.cc:1434] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:09.225133 1172 tablet_bootstrap.cc:492] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.227029 1172 tablet_bootstrap.cc:654] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:09.212496 854 leader_election.cc:343] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.237126 2392 raft_consensus.cc:493] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.237268 2392 raft_consensus.cc:515] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.237815 2392 leader_election.cc:290] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.238380 778 raft_consensus.cc:3060] T 036d306d592f41cb96814aa60e30551e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:09.240046 1172 tablet_bootstrap.cc:492] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.240180 1172 ts_tablet_manager.cc:1403] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.005s
I20260505 07:54:09.240995 1172 raft_consensus.cc:359] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.241178 1172 raft_consensus.cc:385] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.241247 1172 raft_consensus.cc:740] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.241456 1172 consensus_queue.cc:260] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.242008 1172 ts_tablet_manager.cc:1434] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:09.242281 1172 tablet_bootstrap.cc:492] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.244338 1172 tablet_bootstrap.cc:654] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.245108 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 840eebf9e7404bf7807b7e51996957e5 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]) successfully deleted
I20260505 07:54:09.245846 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4e5af08ddc4c77b5640b8295b236b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:09.249188 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e0c8ad716e443588d4ae3aa28ed4bf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:09.249801 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 02716f62aa354d8f839266661b7f05ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02716f62aa354d8f839266661b7f05ea already in progress: creating tablet
W20260505 07:54:09.250239 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet abaa596871bb4a82aefc317932e91d35 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abaa596871bb4a82aefc317932e91d35 already in progress: creating tablet
I20260505 07:54:09.252945 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7001482d90364dbdb8b9c5bfb98e1cd4" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:09.253170 919 raft_consensus.cc:2393] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
W20260505 07:54:09.253688 854 leader_election.cc:343] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.253046 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca12a17fb014374bf7ed764a91e6985" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:09.256924 2445 tablet.cc:1724] T 08c6ed755fc64080a2310fe9a4e0a68b P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:09.257202 2392 raft_consensus.cc:493] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.257313 2392 raft_consensus.cc:515] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.257907 2392 leader_election.cc:290] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.258796 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e010f23c4df647ab863e03c3cfed7898" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:09.259285 854 leader_election.cc:343] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.259625 1172 tablet_bootstrap.cc:492] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.259763 1172 ts_tablet_manager.cc:1403] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.002s
I20260505 07:54:09.263196 1172 raft_consensus.cc:359] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.263377 1172 raft_consensus.cc:385] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.263427 1172 raft_consensus.cc:740] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.265698 1172 consensus_queue.cc:260] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.268584 2392 raft_consensus.cc:493] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.268656 1172 ts_tablet_manager.cc:1434] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260505 07:54:09.268733 2392 raft_consensus.cc:515] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.269304 2392 leader_election.cc:290] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.269552 2447 tablet_replica.cc:333] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:09.269744 1378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 3 candidate_status { last_received { term: 2 index: 686 } } ignore_live_leader: false dest_uuid: "23c57a2c927c4c54bccecfeb3a775311" is_pre_election: true
I20260505 07:54:09.269855 1350 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 106) took 1626 ms. Trace:
W20260505 07:54:09.270107 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 72da22d790624b07bfca6f6b6301c925 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72da22d790624b07bfca6f6b6301c925 already in progress: creating tablet
I20260505 07:54:09.270381 1172 tablet_bootstrap.cc:492] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.271600 2276 ts_tablet_manager.cc:1929] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:09.271751 2276 log.cc:1199] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/26f3544e74d941d286a2a0317b941ab6
I20260505 07:54:09.272082 2276 ts_tablet_manager.cc:1950] T 26f3544e74d941d286a2a0317b941ab6 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:09.272178 1172 tablet_bootstrap.cc:654] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.272521 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 62) took 1654 ms. Trace:
I20260505 07:54:09.272357 1349 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 107) took 1628 ms. Trace:
I20260505 07:54:09.272647 2276 rpcz_store.cc:276] 0505 07:54:07.618368 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.618761 (+ 393us) service_pool.cc:224] Handling call
0505 07:54:08.887626 (+1268865us) ts_tablet_manager.cc:1151] Deleting tablet 26f3544e74d941d286a2a0317b941ab6
0505 07:54:08.887634 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:08.941866 (+ 54232us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.271519 (+329653us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.272505 (+ 986us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1262812}
I20260505 07:54:09.273101 2276 tablet_replica.cc:333] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:09.273319 2276 raft_consensus.cc:2243] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:09.272938 1349 rpcz_store.cc:276] 0505 07:54:07.643477 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.643562 (+ 85us) service_pool.cc:224] Handling call
0505 07:54:07.689120 (+ 45558us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:07.704048 (+ 14928us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:07.704226 (+ 178us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:07.704331 (+ 105us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:07.766559 (+ 62228us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:09.272299 (+1505740us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:07.705018 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:07.705118 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:07.705124 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:07.705126 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:07.705251 (+ 125us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:07.705266 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:07.705270 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:07.705404 (+ 134us) tablet.cc:701] Row locks acquired
0505 07:54:07.705407 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:07.705443 (+ 36us) write_op.cc:270] Start()
0505 07:54:07.705477 (+ 34us) write_op.cc:276] Timestamp: P: 1777967647705416 usec, L: 0
0505 07:54:07.705481 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:07.705669 (+ 188us) log.cc:844] Serialized 405 byte log entry
0505 07:54:07.714634 (+ 8965us) op_driver.cc:464] REPLICATION: finished
0505 07:54:07.715278 (+ 644us) write_op.cc:301] APPLY: starting
0505 07:54:07.715298 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:07.715349 (+ 51us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:07.765742 (+ 50393us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:07.765817 (+ 75us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:07.765856 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:07.765876 (+ 20us) write_op.cc:312] APPLY: finished
0505 07:54:07.765957 (+ 81us) log.cc:844] Serialized 23 byte log entry
0505 07:54:07.766029 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.766049 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:07.766056 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:07.766125 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.766128 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":570,"apply.run_cpu_time_us":513,"apply.run_wall_time_us":50844,"num_ops":1,"prepare.queue_time_us":66,"prepare.run_cpu_time_us":752,"prepare.run_wall_time_us":752,"raft.queue_time_us":400,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":188,"replication_time_us":9118}]]}
I20260505 07:54:09.273490 2276 raft_consensus.cc:2272] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.273769 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_0f05cd371317454b9be3fe9d7c31a9c4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:09.274092 2276 ts_tablet_manager.cc:1916] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:09.274320 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0f05cd371317454b9be3fe9d7c31a9c4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:09.276667 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7d8265c05b74b28959d0de41fad1fde" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.277115 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 26f3544e74d941d286a2a0317b941ab6 (table default.Table_82c755fca0b64337a44a63876e04fa68 [id=ded6ca9533da4aa197bbd8c9b4a72066]) successfully deleted
W20260505 07:54:09.278409 854 leader_election.cc:343] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.269960 1350 rpcz_store.cc:276] 0505 07:54:07.643328 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.643387 (+ 59us) service_pool.cc:224] Handling call
0505 07:54:07.705057 (+ 61670us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:07.705305 (+ 248us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:07.705477 (+ 172us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:07.705558 (+ 81us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:07.768637 (+ 63079us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:09.269768 (+1501131us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:07.706101 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:07.706211 (+ 110us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:07.706219 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:07.706223 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:07.706318 (+ 95us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:07.706330 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:07.706337 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:07.706400 (+ 63us) tablet.cc:701] Row locks acquired
0505 07:54:07.706404 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:07.706436 (+ 32us) write_op.cc:270] Start()
0505 07:54:07.706471 (+ 35us) write_op.cc:276] Timestamp: P: 1777967647706411 usec, L: 0
0505 07:54:07.706476 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:07.706647 (+ 171us) log.cc:844] Serialized 405 byte log entry
0505 07:54:07.718847 (+ 12200us) op_driver.cc:464] REPLICATION: finished
0505 07:54:07.766196 (+ 47349us) write_op.cc:301] APPLY: starting
0505 07:54:07.766210 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:07.766254 (+ 44us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:07.766288 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:07.766334 (+ 46us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:07.766372 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:07.766383 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:07.767661 (+ 1278us) log.cc:844] Serialized 23 byte log entry
0505 07:54:07.767722 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.767744 (+ 22us) write_op.cc:454] Released schema lock
0505 07:54:07.767753 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":47307,"apply.run_cpu_time_us":436,"apply.run_wall_time_us":1624,"num_ops":1,"prepare.queue_time_us":50,"prepare.run_cpu_time_us":618,"prepare.run_wall_time_us":617,"replication_time_us":12338,"thread_start_us":128937,"threads_started":1}]]}
I20260505 07:54:09.279558 1350 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_54f371d41b5e4a54b08bb82a834ac0af" } modify_external_catalogs: true
I20260505 07:54:09.279801 1350 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_54f371d41b5e4a54b08bb82a834ac0af" } modify_external_catalogs: true
I20260505 07:54:09.115010 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66082a7a75e345639e7a268727922fbb" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.161777 2105 raft_consensus.cc:2804] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:09.283838 2105 raft_consensus.cc:493] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:09.283951 2105 raft_consensus.cc:3060] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:09.284302 1172 tablet_bootstrap.cc:492] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.284483 1172 ts_tablet_manager.cc:1403] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260505 07:54:09.285478 2447 raft_consensus.cc:2243] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:09.285432 1172 raft_consensus.cc:359] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.285614 1172 raft_consensus.cc:385] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.285687 1172 raft_consensus.cc:740] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.286597 2105 raft_consensus.cc:515] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.287177 2105 leader_election.cc:290] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:09.287518 2268 raft_consensus.cc:2243] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:09.287748 2268 raft_consensus.cc:2272] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.288286 2268 ts_tablet_manager.cc:1916] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:09.221491 989 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.0.105.62:33621, user_credentials={real_user=slave}} blocked reactor thread for 71932.8us
I20260505 07:54:09.290313 2450 leader_election.cc:290] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565), 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401)
W20260505 07:54:09.160392 988 leader_election.cc:343] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:09.292373 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 131985us
I20260505 07:54:09.292610 1172 consensus_queue.cc:260] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.293218 1172 ts_tablet_manager.cc:1434] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260505 07:54:09.293337 2302 consensus_queue.cc:1048] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.293536 1172 tablet_bootstrap.cc:492] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.295266 1176 tablet_bootstrap.cc:492] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:09.295409 1176 ts_tablet_manager.cc:1403] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.361s user 0.024s sys 0.095s
I20260505 07:54:09.296427 1176 raft_consensus.cc:359] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.296636 1176 raft_consensus.cc:385] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.296691 1176 raft_consensus.cc:740] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.296960 1176 consensus_queue.cc:260] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.297708 1176 ts_tablet_manager.cc:1434] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260505 07:54:09.298497 2447 raft_consensus.cc:2272] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.298532 1176 tablet_bootstrap.cc:492] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:09.299134 2447 ts_tablet_manager.cc:1916] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:09.300014 988 consensus_peers.cc:597] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:09.301004 988 leader_election.cc:343] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 840eebf9e7404bf7807b7e51996957e5
I20260505 07:54:09.301184 988 leader_election.cc:304] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:09.301401 988 raft_consensus.cc:2686] T 840eebf9e7404bf7807b7e51996957e5 P d63a3e535f8e4dd4a88e447253405b77: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260505 07:54:09.301721 988 leader_election.cc:343] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.302106 2302 consensus_queue.cc:1048] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.303740 1181 tablet_bootstrap.cc:492] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:09.303903 1181 ts_tablet_manager.cc:1403] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.218s user 0.000s sys 0.004s
I20260505 07:54:09.304036 1172 tablet_bootstrap.cc:654] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.304951 1181 raft_consensus.cc:359] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.305213 1181 raft_consensus.cc:385] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.305326 1181 raft_consensus.cc:740] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.305593 1181 consensus_queue.cc:260] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.306399 1181 ts_tablet_manager.cc:1434] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:09.306969 1181 tablet_bootstrap.cc:492] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:09.307993 1172 tablet_bootstrap.cc:492] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.308164 1172 ts_tablet_manager.cc:1403] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.001s
I20260505 07:54:09.309041 1172 raft_consensus.cc:359] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.309227 1172 raft_consensus.cc:385] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.309289 1172 raft_consensus.cc:740] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.310257 759 tablet_service.cc:1558] Processing DeleteTablet for tablet a9494c3b6a99487596a030856b371b68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6fe19306f7df49fdb771d21f6560e697) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.310505 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.309486 1172 consensus_queue.cc:260] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.312875 1172 ts_tablet_manager.cc:1434] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:09.313077 1172 tablet_bootstrap.cc:492] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.314253 1172 tablet_bootstrap.cc:654] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.315946 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7001482d90364dbdb8b9c5bfb98e1cd4" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.316844 759 tablet_service.cc:1558] Processing DeleteTablet for tablet a9b7c6e9463342a5a73af12da4447f89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.319602 2469 tablet.cc:1724] T 1001dd888ecd4166ba217506739f71e8 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:09.324823 2392 raft_consensus.cc:493] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.324988 2392 raft_consensus.cc:515] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.325672 2392 leader_election.cc:290] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.326637 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa049ad4edbd466a860428c822a52724" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.327410 1176 tablet_bootstrap.cc:654] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:09.327878 854 leader_election.cc:343] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.326201 1353 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3] alter complete (version 3)
I20260505 07:54:09.330531 1348 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 112) took 1662 ms. Trace:
I20260505 07:54:09.330674 1348 rpcz_store.cc:276] 0505 07:54:07.668482 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.045047 (+1376565us) service_pool.cc:224] Handling call
0505 07:54:09.270200 (+225153us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:09.270761 (+ 561us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:09.330224 (+ 59463us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:09.330353 (+ 129us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:09.330482 (+ 129us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.282263 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.282365 (+ 102us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.282371 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:09.282374 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:09.282447 (+ 73us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.282457 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.282461 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.282519 (+ 58us) tablet.cc:701] Row locks acquired
0505 07:54:09.282521 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:09.282548 (+ 27us) write_op.cc:270] Start()
0505 07:54:09.282574 (+ 26us) write_op.cc:276] Timestamp: P: 1777967649282518 usec, L: 0
0505 07:54:09.282577 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.282719 (+ 142us) log.cc:844] Serialized 433 byte log entry
0505 07:54:09.314122 (+ 31403us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.323887 (+ 9765us) write_op.cc:301] APPLY: starting
0505 07:54:09.323900 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.323969 (+ 69us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.323998 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.324123 (+ 125us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.324162 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:09.324173 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:09.324245 (+ 72us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.329955 (+ 5710us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.329982 (+ 27us) write_op.cc:454] Released schema lock
0505 07:54:09.329991 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":9720,"num_ops":2,"prepare.queue_time_us":10977,"raft.queue_time_us":2941,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"replication_time_us":31518,"threads_started":2}]]}
I20260505 07:54:09.341271 717 leader_election.cc:304] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:09.341605 1932 raft_consensus.cc:3060] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:09.258699 1034 consensus_meta.cc:287] T db430ab084354b48aceb445ff3306ef9 P d63a3e535f8e4dd4a88e447253405b77: Time spent flushing consensus metadata: real 1.038s user 0.023s sys 0.000s
I20260505 07:54:09.259897 2308 consensus_queue.cc:1048] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.345909 1100 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
W20260505 07:54:09.349117 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 88994.4us
W20260505 07:54:09.349725 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 47681.8us
I20260505 07:54:09.350716 2392 raft_consensus.cc:493] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.350855 2392 raft_consensus.cc:515] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.351395 2392 leader_election.cc:290] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.355234 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472441bd29704be59aabb0f7a4964bb0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.355484 2307 consensus_queue.cc:1048] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:09.355943 854 leader_election.cc:343] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.356990 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 with delete_type TABLET_DATA_DELETED (replaced by tablet 66082a7a75e345639e7a268727922fbb) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.357424 756 tablet_service.cc:1558] Processing DeleteTablet for tablet e82e9ae417784b7e83f4843b129d0580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:09.359038 988 leader_election.cc:343] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.360610 2302 consensus_queue.cc:1048] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:09.362761 988 leader_election.cc:343] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:09.369233 988 leader_election.cc:343] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:09.370499 988 consensus_peers.cc:597] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:09.371130 2307 consensus_queue.cc:1048] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:09.371713 988 leader_election.cc:343] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.372167 2307 consensus_queue.cc:1048] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:09.372769 988 consensus_peers.cc:597] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:09.373008 988 consensus_peers.cc:597] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:09.374243 988 leader_election.cc:343] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:09.374902 988 consensus_peers.cc:597] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:09.375530 988 leader_election.cc:343] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:09.363682 991 leader_election.cc:343] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 32c008bd99224a7595cecc607e5496a9
I20260505 07:54:09.376024 991 leader_election.cc:304] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.369782 2473 tablet.cc:1724] T 08c6ed755fc64080a2310fe9a4e0a68b P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:09.383826 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.383983 746 tablet_service.cc:1558] Processing DeleteTablet for tablet aab3850799254dda9716fd5bf9404159 with delete_type TABLET_DATA_DELETED (replaced by tablet 5006965919824adbbb23d744fb9ff335) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.384017 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 02e06d8815034d91986cef12964860ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.384642 1932 raft_consensus.cc:2749] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.384812 752 tablet_service.cc:1558] Processing DeleteTablet for tablet abe5f6f0774545b296bf1b03a08e145d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.389596 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 472441bd29704be59aabb0f7a4964bb0 with delete_type TABLET_DATA_DELETED (replaced by tablet e7887c3462f74740b314e3bec36e280d) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.389868 746 tablet_service.cc:1558] Processing DeleteTablet for tablet df1b62a30b2842b897a04c793bb5e7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.390332 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 297537a92684425d8508f7b3b4c9d379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.390753 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 01431ad112e8445487823efe8feb0dbe) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.391149 746 tablet_service.cc:1558] Processing DeleteTablet for tablet ae829daad64545e687a9528e5c38b9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.391492 746 tablet_service.cc:1558] Processing DeleteTablet for tablet f200abef13ac42698a9b96fc8af82ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.391942 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 54e4f7f895d943639653c8c32724baf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.392412 756 tablet_service.cc:1558] Processing DeleteTablet for tablet accb2dedc30e494ebd22c33594cee661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.392915 746 tablet_service.cc:1558] Processing DeleteTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.393297 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 872362aecf6a44b59e586b1fa7637c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.393676 746 tablet_service.cc:1558] Processing DeleteTablet for tablet e7887c3462f74740b314e3bec36e280d with delete_type TABLET_DATA_DELETED (replaced by tablet fba1d19f902a4a0a8b581135f6611129) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.394070 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 727cacd242844455915bd9963d2db19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.394459 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c4fc18f6faa488882d9bffb30793cdd) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.394891 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ad8a79245ad47738d079752e51aed2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.395296 746 tablet_service.cc:1558] Processing DeleteTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.395678 756 tablet_service.cc:1558] Processing DeleteTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.395953 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 02716f62aa354d8f839266661b7f05ea with delete_type TABLET_DATA_DELETED (replaced by tablet 9f541eff399641dd81a0771186805493) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:09.396469 2257 log.cc:927] Time spent T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Append to log took a long time: real 0.094s user 0.000s sys 0.000s
I20260505 07:54:09.396463 746 tablet_service.cc:1558] Processing DeleteTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.398736 1181 tablet_bootstrap.cc:654] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:09.399296 899 consensus_meta.cc:287] T 9926e569032144ec9d3148f4561333ab P d2799f5c6b4f4336b976ca9bbb458533: Time spent flushing consensus metadata: real 0.885s user 0.020s sys 0.000s
I20260505 07:54:09.403757 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_3f6be0e718b349a089b5aa8436614766"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:09.404484 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f6be0e718b349a089b5aa8436614766 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:09.401764 1034 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:47472 (request call id 83) took 1186 ms. Trace:
I20260505 07:54:09.414113 1034 rpcz_store.cc:276] 0505 07:54:08.215676 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:08.215725 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:08.216080 (+ 355us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0505 07:54:08.216113 (+ 33us) ts_tablet_manager.cc:682] Creating new metadata...
0505 07:54:08.221003 (+ 4890us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.342763 (+1121760us) tablet_replica.cc:206] Creating consensus instance
0505 07:54:09.401746 (+ 58983us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:09.414630 2307 consensus_queue.cc:1048] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.415086 2302 raft_consensus.cc:2804] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:09.415246 2302 raft_consensus.cc:697] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:09.415524 2302 consensus_queue.cc:237] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.418850 991 leader_election.cc:304] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.420246 1878 raft_consensus.cc:2804] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:09.420710 1878 raft_consensus.cc:697] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:09.421136 2392 raft_consensus.cc:493] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.421272 2392 raft_consensus.cc:515] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.421921 2392 leader_election.cc:290] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.422401 1878 consensus_queue.cc:237] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.422401 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7fe9c7199cc4039ae877dc3d4aab17e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:09.426481 854 leader_election.cc:343] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.428552 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 080b2984fcf2478a8def6540da63554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.429275 746 tablet_service.cc:1558] Processing DeleteTablet for tablet d622612bdc404727a27b83769bf39738 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.431983 746 tablet_service.cc:1558] Processing DeleteTablet for tablet e010f23c4df647ab863e03c3cfed7898 with delete_type TABLET_DATA_DELETED (replaced by tablet 09019fdf60374b11bf3427f21a55b5e0) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.432166 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.433313 1172 tablet_bootstrap.cc:492] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.433454 1172 ts_tablet_manager.cc:1403] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.120s user 0.042s sys 0.050s
I20260505 07:54:09.434212 1172 raft_consensus.cc:359] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.434383 1172 raft_consensus.cc:385] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.434439 1172 raft_consensus.cc:740] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.434643 1172 consensus_queue.cc:260] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.435325 1172 ts_tablet_manager.cc:1434] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:09.435685 1172 tablet_bootstrap.cc:492] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.436087 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 with delete_type TABLET_DATA_DELETED (replaced by tablet 4af9def97a454f3f9b63d637db42c110) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.437281 1172 tablet_bootstrap.cc:654] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.442055 2308 consensus_queue.cc:1048] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.445308 1354 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 113) took 1773 ms. Trace:
I20260505 07:54:09.445436 1354 rpcz_store.cc:276] 0505 07:54:07.672224 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.070726 (+1398502us) service_pool.cc:224] Handling call
0505 07:54:09.271309 (+200583us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:09.271750 (+ 441us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:09.444999 (+173249us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:09.445124 (+ 125us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:09.445229 (+ 105us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.368458 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.368591 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.368598 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:09.368601 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:09.368701 (+ 100us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.368715 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.368720 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.368807 (+ 87us) tablet.cc:701] Row locks acquired
0505 07:54:09.368810 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:09.368845 (+ 35us) write_op.cc:270] Start()
0505 07:54:09.368880 (+ 35us) write_op.cc:276] Timestamp: P: 1777967649368805 usec, L: 0
0505 07:54:09.368884 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.408286 (+ 39402us) log.cc:844] Serialized 433 byte log entry
0505 07:54:09.409160 (+ 874us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65bae8f22 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac
0505 07:54:09.443401 (+ 34241us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.443610 (+ 209us) write_op.cc:301] APPLY: starting
0505 07:54:09.443627 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.443675 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.443679 (+ 4us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.443797 (+ 118us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.443832 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:09.443839 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:09.443915 (+ 76us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.444007 (+ 92us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.444027 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:09.444036 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.444121 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.444125 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":135,"apply.run_cpu_time_us":500,"apply.run_wall_time_us":500,"num_ops":2,"prepare.queue_time_us":96265,"prepare.run_cpu_time_us":1635,"prepare.run_wall_time_us":40774,"replication_time_us":74466,"spinlock_wait_cycles":89545088}]]}
I20260505 07:54:09.448124 1349 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 114) took 1768 ms. Trace:
I20260505 07:54:09.448405 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.448235 1349 rpcz_store.cc:276] 0505 07:54:07.679736 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.273557 (+1593821us) service_pool.cc:224] Handling call
0505 07:54:09.274407 (+ 850us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:09.274903 (+ 496us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:09.447834 (+172931us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:09.447945 (+ 111us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:09.448045 (+ 100us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.409352 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.409577 (+ 225us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.409586 (+ 9us) write_op.cc:435] Acquired schema lock
0505 07:54:09.409614 (+ 28us) tablet.cc:662] Decoding operations
0505 07:54:09.409819 (+ 205us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.409835 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.409842 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.410029 (+ 187us) tablet.cc:701] Row locks acquired
0505 07:54:09.410033 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:09.410099 (+ 66us) write_op.cc:270] Start()
0505 07:54:09.410165 (+ 66us) write_op.cc:276] Timestamp: P: 1777967649410050 usec, L: 0
0505 07:54:09.410171 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.410485 (+ 314us) log.cc:844] Serialized 433 byte log entry
0505 07:54:09.443482 (+ 32997us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.444198 (+ 716us) write_op.cc:301] APPLY: starting
0505 07:54:09.444212 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.444252 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.444289 (+ 37us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.444384 (+ 95us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.444434 (+ 50us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:09.444443 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:09.444509 (+ 66us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.444565 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.444581 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:09.444588 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":677,"apply.run_cpu_time_us":456,"apply.run_wall_time_us":456,"num_ops":2,"prepare.queue_time_us":134033,"prepare.run_cpu_time_us":988,"prepare.run_wall_time_us":1593,"replication_time_us":33289}]]}
I20260505 07:54:09.451318 2407 consensus_queue.cc:1048] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.451853 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 93d878ae0ddb42319560f74e0a490321 with delete_type TABLET_DATA_DELETED (replaced by tablet bc9ae6404dd64bd8bf4898768e94e4f1) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.452244 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 85d485ae2d1d4d76a6a5d988cec8291a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.452606 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.452937 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 12501fdf2cc5491cb1a06db490e70810 with delete_type TABLET_DATA_DELETED (replaced by tablet d622612bdc404727a27b83769bf39738) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.453259 756 tablet_service.cc:1558] Processing DeleteTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e08cf03ea0b495a94c6566ea6c8caad) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.453552 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 with delete_type TABLET_DATA_DELETED (replaced by tablet 42360f8ee4ff47c2af8d287f792207de) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.453856 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 17327cee69a24e0cb493ff55f948bf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.454157 756 tablet_service.cc:1558] Processing DeleteTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf with delete_type TABLET_DATA_DELETED (replaced by tablet 3a73dbb979344805876efbafbf1b2985) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.454434 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.454762 756 tablet_service.cc:1558] Processing DeleteTablet for tablet d896e290167b4d0081e8e8b831c8ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.455116 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.455403 756 tablet_service.cc:1558] Processing DeleteTablet for tablet e8bf49b787ef4de095448751464708a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.459851 2447 ts_tablet_manager.cc:1929] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:09.460093 2447 log.cc:1199] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/f88a5c0d0e2c436c8d8b6afc1f3c2edf
I20260505 07:54:09.460609 2447 ts_tablet_manager.cc:1950] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:09.464072 915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727cacd242844455915bd9963d2db19f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:09.464263 915 raft_consensus.cc:3060] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:09.467089 915 raft_consensus.cc:2468] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:09.474356 2447 tablet_replica.cc:333] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:09.474678 2447 raft_consensus.cc:2243] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:09.475178 2447 raft_consensus.cc:2272] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.475967 2447 ts_tablet_manager.cc:1916] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:09.477593 1348 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 116) took 1765 ms. Trace:
I20260505 07:54:09.477747 1348 rpcz_store.cc:276] 0505 07:54:07.712063 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.331473 (+1619410us) service_pool.cc:224] Handling call
0505 07:54:09.403669 (+ 72196us) spinlock_profiling.cc:243] Waited 71.8 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d6b13 00007fd65a46a7c6 00007fd65a46a4f5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:09.431820 (+ 28151us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:09.432326 (+ 506us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:09.477303 (+ 44977us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:09.477421 (+ 118us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:09.477526 (+ 105us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.432989 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.433094 (+ 105us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.433101 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:09.433105 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:09.433191 (+ 86us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.433200 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.433204 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.433269 (+ 65us) tablet.cc:701] Row locks acquired
0505 07:54:09.433271 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:09.433303 (+ 32us) write_op.cc:270] Start()
0505 07:54:09.433338 (+ 35us) write_op.cc:276] Timestamp: P: 1777967649433256 usec, L: 0
0505 07:54:09.433342 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.433506 (+ 164us) log.cc:844] Serialized 433 byte log entry
0505 07:54:09.474710 (+ 41204us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.476732 (+ 2022us) write_op.cc:301] APPLY: starting
0505 07:54:09.476749 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.476791 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.476819 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.477003 (+ 184us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.477042 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:09.477051 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:09.477115 (+ 64us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.477165 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.477181 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:09.477186 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":165070720,"child_traces":[["op",{"apply.queue_time_us":1951,"apply.run_cpu_time_us":517,"apply.run_wall_time_us":517,"num_ops":2,"prepare.queue_time_us":151,"prepare.run_cpu_time_us":607,"prepare.run_wall_time_us":1281,"replication_time_us":41334}]]}
I20260505 07:54:09.480139 1176 tablet_bootstrap.cc:492] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:09.480347 1176 ts_tablet_manager.cc:1403] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.182s user 0.005s sys 0.009s
I20260505 07:54:09.480480 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_72dc49a729784b0cb6f4b62ee11fa2d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:09.481186 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_72dc49a729784b0cb6f4b62ee11fa2d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:09.481583 756 tablet_service.cc:1558] Processing DeleteTablet for tablet a7d8265c05b74b28959d0de41fad1fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.482028 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d83e9d4243e409794ffe8a342bce6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.482313 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 7585f00119e14006990ed551667b4486 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.482625 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 976359601ad74b149696dc7f555b550c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.482975 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d6494590052459485059f5dce46f34f with delete_type TABLET_DATA_DELETED (replaced by tablet c822fafe72544ca99fde910c8e764755) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.483302 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.483942 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_98bcfc12727141509f5ccf7049009989"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:09.484562 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_98bcfc12727141509f5ccf7049009989 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:09.488440 1353 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 994dbc9aa33248ee991dcf76018fe520 consensus state has no leader
I20260505 07:54:09.490927 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.491472 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 003cd4472b424aacbba8c5f89900e228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.491961 756 tablet_service.cc:1558] Processing DeleteTablet for tablet bd581a6799a14a8dafb80d411e8936f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.492177 2449 tablet.cc:1724] T 1001dd888ecd4166ba217506739f71e8 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:09.492480 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 840eebf9e7404bf7807b7e51996957e5 with delete_type TABLET_DATA_DELETED (replaced by tablet bf0ce3e6c00541569d15c1b6ef347579) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.493521 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e707143a4e64725aa66db05b24999a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.494163 1176 raft_consensus.cc:359] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.494438 1176 raft_consensus.cc:385] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.494549 1176 raft_consensus.cc:740] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.494843 1176 consensus_queue.cc:260] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:09.509781 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 133940us
I20260505 07:54:09.515388 2392 raft_consensus.cc:493] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.515533 2392 raft_consensus.cc:515] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.516168 2392 leader_election.cc:290] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.512095 2307 raft_consensus.cc:493] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.519707 2307 raft_consensus.cc:515] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.520295 2307 leader_election.cc:290] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:09.524617 2392 raft_consensus.cc:493] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.524740 2392 raft_consensus.cc:515] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.525313 2392 leader_election.cc:290] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.525812 2307 consensus_queue.cc:1048] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.517968 1181 tablet_bootstrap.cc:492] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:09.529096 1181 ts_tablet_manager.cc:1403] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.222s user 0.001s sys 0.006s
I20260505 07:54:09.531177 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:09.532606 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:09.533449 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:09.534323 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:09.535351 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:09.536170 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:09.536711 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:09.537287 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet df3a49b59ed6485f970074391f932482 (table default.Table_7af623f581af498b86c8c3b1adbc6b80 [id=306149ae47a745edabf863d5004d5012]). Expected version 1 got 0
I20260505 07:54:09.537801 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:09.539474 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:09.540284 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:09.540928 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:09.541366 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:09.541790 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
W20260505 07:54:09.541904 856 leader_election.cc:343] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.542047 856 leader_election.cc:304] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.542285 2444 raft_consensus.cc:2749] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.542513 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet abe5f6f0774545b296bf1b03a08e145d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.542820 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:09.543648 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:09.544504 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:09.545230 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:09.546120 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
I20260505 07:54:09.546175 2276 ts_tablet_manager.cc:1929] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:09.546273 2276 log.cc:1199] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e278b86e7b5d444ebad8d0f6921d4c62
I20260505 07:54:09.546684 2276 ts_tablet_manager.cc:1950] T e278b86e7b5d444ebad8d0f6921d4c62 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:09.546968 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:09.547451 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_728dc87e78954a7fbbf495d74705e382 [id=2c06ff8eed31432e87f08e9b59049652]). Expected version 2 got 0
I20260505 07:54:09.547495 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 61) took 1942 ms. Trace:
I20260505 07:54:09.548270 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:09.548270 1181 raft_consensus.cc:359] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.548447 1181 raft_consensus.cc:385] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.548512 1181 raft_consensus.cc:740] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.548714 1181 consensus_queue.cc:260] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.549450 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:09.550019 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.550663 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:09.551421 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:09.552172 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:09.552062 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet aab3850799254dda9716fd5bf9404159 with delete_type TABLET_DATA_DELETED (replaced by tablet 5006965919824adbbb23d744fb9ff335) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.553220 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:09.553959 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.554340 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:09.555472 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:09.555994 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (table default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c [id=8a910f6a553a47ea87ec170e376982d1]) successfully deleted
I20260505 07:54:09.495684 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 1852f7f4ff594639908b03730b68fe86 with delete_type TABLET_DATA_DELETED (replaced by tablet aab3850799254dda9716fd5bf9404159) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.516542 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c837877946604ef29f95dac9dd449bf3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.526324 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f106a6d4855148fe83b1e90ea9f27d39" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.557054 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]). Expected version 2 got 0
I20260505 07:54:09.536711 1932 raft_consensus.cc:493] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.557287 1932 raft_consensus.cc:515] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:09.557955 1932 leader_election.cc:290] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:09.549099 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.549407 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 with delete_type TABLET_DATA_DELETED (replaced by tablet d5083f98e8e14ccf99a19cdf2bc59018) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.549503 759 tablet_service.cc:1558] Processing DeleteTablet for tablet 78a6470720984761a8606abeafbef992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.549602 755 tablet_service.cc:1558] Processing DeleteTablet for tablet 397d33e908af4a0bb8dde26d273be01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.558352 752 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.558385 755 tablet_service.cc:1558] Processing DeleteTablet for tablet ccdde3e5a13e436d9f2328831e155adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.558487 755 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.558620 755 tablet_service.cc:1558] Processing DeleteTablet for tablet 202a237ca580482c98803683a77d96d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.558646 746 tablet_service.cc:1558] Processing DeleteTablet for tablet 03f31efbc98c484c950c70d5e84d2136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.549952 747 tablet_service.cc:1558] Processing DeleteTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.550719 753 tablet_service.cc:1558] Processing DeleteTablet for tablet abaa596871bb4a82aefc317932e91d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.551396 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 72da22d790624b07bfca6f6b6301c925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.551752 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 323269e708964848a94c64e6a781a860 with delete_type TABLET_DATA_DELETED (replaced by tablet e010f23c4df647ab863e03c3cfed7898) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.552155 751 tablet_service.cc:1558] Processing DeleteTablet for tablet e278b86e7b5d444ebad8d0f6921d4c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.552273 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a06f77255544d7dabb71a4f70ada809 with delete_type TABLET_DATA_DELETED (replaced by tablet 12841efcef6041d89fc6d7d9b90448c0) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.552743 749 tablet_service.cc:1558] Processing DeleteTablet for tablet f1058fcb9fa6457a85489049ca1d9ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.550457 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 09b46ff0101a4e529077b9e30492ace7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.553011 758 tablet_service.cc:1558] Processing DeleteTablet for tablet b12d2d2b866348d781209fa787da7d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.553344 744 tablet_service.cc:1558] Processing DeleteTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.553109 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 8eb7eaff8837485a8ae3ccb44c60471d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.554340 743 tablet_service.cc:1558] Processing DeleteTablet for tablet 06904a54b1c5463da31af0bc00fb606f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.554059 750 tablet_service.cc:1558] Processing DeleteTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.554913 748 tablet_service.cc:1558] Processing DeleteTablet for tablet 60772a9697ba417b879dc6025eeb8eec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.554461 745 tablet_service.cc:1558] Processing DeleteTablet for tablet 26f3544e74d941d286a2a0317b941ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.560490 2268 ts_tablet_manager.cc:1929] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:09.560508 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:09.560619 2268 log.cc:1199] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/8eb7eaff8837485a8ae3ccb44c60471d
I20260505 07:54:09.560627 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727cacd242844455915bd9963d2db19f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:09.560801 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b404e50ae092419c9396772b3b14eeaa (table default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb]). Expected version 4 got 0
I20260505 07:54:09.561113 2268 ts_tablet_manager.cc:1950] T 8eb7eaff8837485a8ae3ccb44c60471d P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:09.561151 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 08c6ed755fc64080a2310fe9a4e0a68b (table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b]). Expected version 3 got 0
I20260505 07:54:09.561375 2445 tablet.cc:1724] T b404e50ae092419c9396772b3b14eeaa P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260505 07:54:09.561571 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:09.561618 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.562099 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:09.565613 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:09.566256 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]). Expected version 1 got 0
I20260505 07:54:09.566912 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 872362aecf6a44b59e586b1fa7637c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.566987 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:09.567992 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:09.568457 1176 ts_tablet_manager.cc:1434] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.088s user 0.002s sys 0.000s
I20260505 07:54:09.568866 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:09.569509 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]). Expected version 2 got 0
I20260505 07:54:09.571048 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 21480c42b73e4bf0ab2453b085a867c0 (table default.Table_925028e721e24ba68f31fe1040d4c2fd [id=0faa6676bab04919984b580299143271]). Expected version 1 got 0
I20260505 07:54:09.573127 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 01431ad112e8445487823efe8feb0dbe) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.573379 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:09.573540 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet df1b62a30b2842b897a04c793bb5e7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.574242 1181 ts_tablet_manager.cc:1434] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.040s user 0.000s sys 0.002s
I20260505 07:54:09.574561 1181 tablet_bootstrap.cc:492] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:09.576764 1181 tablet_bootstrap.cc:654] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.579633 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 297537a92684425d8508f7b3b4c9d379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.580101 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 080b2984fcf2478a8def6540da63554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.580816 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet f200abef13ac42698a9b96fc8af82ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.581246 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 472441bd29704be59aabb0f7a4964bb0 with delete_type TABLET_DATA_DELETED (replaced by tablet e7887c3462f74740b314e3bec36e280d) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.585461 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet e7887c3462f74740b314e3bec36e280d with delete_type TABLET_DATA_DELETED (replaced by tablet fba1d19f902a4a0a8b581135f6611129) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.586112 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ad8a79245ad47738d079752e51aed2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.586611 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.591022 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet e010f23c4df647ab863e03c3cfed7898 with delete_type TABLET_DATA_DELETED (replaced by tablet 09019fdf60374b11bf3427f21a55b5e0) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.591435 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet ccdde3e5a13e436d9f2328831e155adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.591921 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 7585f00119e14006990ed551667b4486 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.592363 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d6494590052459485059f5dce46f34f with delete_type TABLET_DATA_DELETED (replaced by tablet c822fafe72544ca99fde910c8e764755) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.593036 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 40) took 2510 ms. Trace:
I20260505 07:54:09.593185 2268 rpcz_store.cc:276] 0505 07:54:07.082548 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.114259 (+ 31711us) service_pool.cc:224] Handling call
0505 07:54:09.221256 (+2106997us) ts_tablet_manager.cc:1151] Deleting tablet 8eb7eaff8837485a8ae3ccb44c60471d
0505 07:54:09.221265 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.404080 (+182815us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.560020 (+155940us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.593019 (+ 32999us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31088,"tablet-delete.queue_time_us":2106900}
I20260505 07:54:09.594012 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 41) took 2511 ms. Trace:
I20260505 07:54:09.594218 2268 rpcz_store.cc:276] 0505 07:54:07.082653 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.114360 (+ 31707us) service_pool.cc:224] Handling call
0505 07:54:09.593805 (+2479445us) ts_tablet_manager.cc:1151] Deleting tablet d896e290167b4d0081e8e8b831c8ae2a
0505 07:54:09.593819 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.593993 (+ 174us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2479333}
I20260505 07:54:09.594794 2268 tablet_replica.cc:333] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:09.596383 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1175f7deec14fe39098ec89ab3bc049" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.596939 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e0c8ad716e443588d4ae3aa28ed4bf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.597398 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e010f23c4df647ab863e03c3cfed7898" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.597879 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7d8265c05b74b28959d0de41fad1fde" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.574235 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a73dbb979344805876efbafbf1b2985" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.598629 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa049ad4edbd466a860428c822a52724" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.599121 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472441bd29704be59aabb0f7a4964bb0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.599615 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7fe9c7199cc4039ae877dc3d4aab17e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.600489 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c837877946604ef29f95dac9dd449bf3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.600953 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f106a6d4855148fe83b1e90ea9f27d39" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.601419 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 with delete_type TABLET_DATA_DELETED (replaced by tablet 4af9def97a454f3f9b63d637db42c110) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.601809 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 840eebf9e7404bf7807b7e51996957e5 with delete_type TABLET_DATA_DELETED (replaced by tablet bf0ce3e6c00541569d15c1b6ef347579) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.604028 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.606129 2438 raft_consensus.cc:493] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.606309 2438 raft_consensus.cc:515] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:09.607019 2438 leader_election.cc:290] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:09.607414 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c4fc18f6faa488882d9bffb30793cdd) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.607756 1176 tablet_bootstrap.cc:492] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:09.607861 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 727cacd242844455915bd9963d2db19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.608371 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 02412333633a4cdebe95a144fa6c363c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.609599 1176 tablet_bootstrap.cc:654] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.612685 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 3075e87e078f4d2aa1d329d18902354b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.613716 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet e29ba55a846548679cb3b86305d0d6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.614044 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 46274a44a06b4306b5dac43c66c85a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.614391 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f207a3ffbc94b01abd9e9d6837f5c81) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.621739 2407 consensus_queue.cc:1048] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.622265 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.622623 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet d3d11bd19f074bbab101448b45426b7a with delete_type TABLET_DATA_DELETED (replaced by tablet 76939b7978ee4f7ea0ad29312de2a31a) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.622963 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 4258df74ea0841449846cdf1f6e00b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.623283 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.623692 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:09.554632 856 leader_election.cc:343] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.627374 856 leader_election.cc:304] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.627849 2392 raft_consensus.cc:2749] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.628101 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 73475.2us
W20260505 07:54:09.628541 856 leader_election.cc:343] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.628644 856 leader_election.cc:304] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:09.628926 856 leader_election.cc:343] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.628998 856 leader_election.cc:304] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.629380 2284 raft_consensus.cc:2749] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.629508 2284 raft_consensus.cc:2749] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.629711 2447 ts_tablet_manager.cc:1929] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:09.558729 899 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:53998 (request call id 164) took 1057 ms. Trace:
W20260505 07:54:09.559878 854 leader_election.cc:343] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.617441 2444 raft_consensus.cc:493] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:09.630383 854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 70518.8us
I20260505 07:54:09.630416 2444 raft_consensus.cc:515] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.630295 899 rpcz_store.cc:276] 0505 07:54:08.501090 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:08.510626 (+ 9536us) service_pool.cc:224] Handling call
0505 07:54:08.511535 (+ 909us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0505 07:54:08.511560 (+ 25us) ts_tablet_manager.cc:682] Creating new metadata...
0505 07:54:08.514480 (+ 2920us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.399485 (+885005us) tablet_replica.cc:206] Creating consensus instance
0505 07:54:09.558709 (+159224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260505 07:54:09.630570 854 leader_election.cc:343] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.630939 2392 raft_consensus.cc:493] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.631031 2392 raft_consensus.cc:515] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.631060 2444 leader_election.cc:290] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.625391 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a73dbb979344805876efbafbf1b2985" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:09.631278 919 raft_consensus.cc:2468] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:09.631493 2392 leader_election.cc:290] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.637888 2276 rpcz_store.cc:276] 0505 07:54:07.604569 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.618785 (+ 14216us) service_pool.cc:224] Handling call
0505 07:54:09.273036 (+1654251us) ts_tablet_manager.cc:1151] Deleting tablet e278b86e7b5d444ebad8d0f6921d4c62
0505 07:54:09.273048 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.543076 (+270028us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.546123 (+ 3047us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.547477 (+ 1354us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1643712}
I20260505 07:54:09.638706 2268 raft_consensus.cc:2243] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:09.639385 2268 pending_rounds.cc:70] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:54:09.639823 2276 tablet_replica.cc:333] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:09.640527 2276 raft_consensus.cc:2243] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:09.640817 2268 pending_rounds.cc:77] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7282555485902569472 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "accb2dedc30e494ebd22c33594cee661" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 1 new_table_name: "default.Table_fa6246cc3ccd4d7082becde923f88828" dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" new_extra_config { } }
W20260505 07:54:09.644101 2268 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47472: Aborted: Op aborted
I20260505 07:54:09.644454 2276 raft_consensus.cc:2272] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.645746 1172 tablet_bootstrap.cc:492] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.645915 1172 ts_tablet_manager.cc:1403] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.210s user 0.037s sys 0.096s
I20260505 07:54:09.646875 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47472 (request call id 63) took 1746 ms. Trace:
W20260505 07:54:09.646854 1315 catalog_manager.cc:5075] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): alter failed for tablet 85d485ae2d1d4d76a6a5d988cec8291a (table default.Table_394242d5ee184ae2b22cc355a2160fb9 [id=6e71d3f928fc45c191f64a945980a107]),no further retry: Not found: Tablet not found: 85d485ae2d1d4d76a6a5d988cec8291a
I20260505 07:54:09.647406 2268 rpcz_store.cc:276] 0505 07:54:07.900920 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.907907 (+ 6987us) service_pool.cc:224] Handling call
0505 07:54:09.646860 (+1738953us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:07.925307 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0505 07:54:07.925416 (+ 109us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0505 07:54:07.925424 (+ 8us) alter_schema_op.cc:66] Acquired schema lock
0505 07:54:07.925431 (+ 7us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0505 07:54:07.925465 (+ 34us) alter_schema_op.cc:128] START. Timestamp: 1777967647925432
0505 07:54:07.925470 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:07.925708 (+ 238us) log.cc:844] Serialized 173 byte log entry
0505 07:54:09.644037 (+1718329us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.644067 (+ 30us) op_driver.cc:389] HandleFailure(Aborted: Op aborted)
0505 07:54:09.644074 (+ 7us) alter_schema_op.cc:171] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":16987,"prepare.run_cpu_time_us":5558,"prepare.run_wall_time_us":111618,"raft.queue_time_us":555,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":7,"replication_time_us":1718521,"thread_start_us":110950,"threads_started":1,"wal-append.queue_time_us":109999,"wal-append.run_cpu_time_us":393,"wal-append.run_wall_time_us":1000456}]]}
I20260505 07:54:09.648618 1172 raft_consensus.cc:359] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.648794 1172 raft_consensus.cc:385] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.648867 1172 raft_consensus.cc:740] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.649092 1172 consensus_queue.cc:260] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.649092 2276 ts_tablet_manager.cc:1916] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:09.649636 1172 ts_tablet_manager.cc:1434] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:09.649942 1172 tablet_bootstrap.cc:492] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.650269 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba2cc218300472cb5fd6dc8cfb7811c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:09.650465 912 raft_consensus.cc:2468] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:09.651698 1172 tablet_bootstrap.cc:654] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:09.651933 856 leader_election.cc:343] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.652083 856 leader_election.cc:304] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:09.652441 856 leader_election.cc:343] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.652597 856 leader_election.cc:304] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.652861 2392 raft_consensus.cc:2749] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.652994 856 consensus_peers.cc:597] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bde59d52fd234343b1ca13a7d4aae6bf. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:09.653039 2392 raft_consensus.cc:2749] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.653126 856 leader_election.cc:343] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.653227 856 leader_election.cc:304] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:09.653299 1315 catalog_manager.cc:5075] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): alter failed for tablet e278b86e7b5d444ebad8d0f6921d4c62 (table default.Table_633da5b39da34f5c88163c9936fd0212 [id=d53c653124364ea5acc357977c9db864]),no further retry: Not found: Tablet not found: e278b86e7b5d444ebad8d0f6921d4c62
I20260505 07:54:09.653498 2392 raft_consensus.cc:2749] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.653892 856 leader_election.cc:343] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.654036 856 leader_election.cc:304] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:09.654322 1353 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet fba1d19f902a4a0a8b581135f6611129 consensus state has no leader
I20260505 07:54:09.654394 2392 raft_consensus.cc:2749] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.654762 856 leader_election.cc:343] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.654850 856 leader_election.cc:304] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.655036 2392 raft_consensus.cc:2749] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.655822 856 leader_election.cc:343] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.655918 856 leader_election.cc:304] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.656114 2392 raft_consensus.cc:2749] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.656759 856 leader_election.cc:343] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.656901 856 leader_election.cc:304] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.657192 2392 raft_consensus.cc:2749] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.657290 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:09.658353 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 8eb7eaff8837485a8ae3ccb44c60471d (table default.Table_66f28ba637f24448816a64c7d316edee [id=266ff796c7a74c23816dd9c422e9d005]) successfully deleted
W20260505 07:54:09.658581 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet d896e290167b4d0081e8e8b831c8ae2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d896e290167b4d0081e8e8b831c8ae2a already in progress: creating tablet
I20260505 07:54:09.659559 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
W20260505 07:54:09.661761 1353 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.1:36755 (request call id 142) took 2096 ms (client timeout 1000 ms). Trace:
I20260505 07:54:09.663272 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 05351965bdba4ccca8d13223cd505948
I20260505 07:54:09.663759 1350 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 115) took 1975 ms. Trace:
I20260505 07:54:09.664379 1350 rpcz_store.cc:276] 0505 07:54:07.688549 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.279336 (+1590787us) service_pool.cc:224] Handling call
0505 07:54:09.279914 (+ 578us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:09.280094 (+ 180us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:09.280164 (+ 70us) catalog_manager.cc:2824] Locking tablets
0505 07:54:09.280308 (+ 144us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:09.474073 (+193765us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:09.474160 (+ 87us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:09.663017 (+188857us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:09.663720 (+ 703us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.411039 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.411238 (+ 199us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.411245 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:09.411248 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:09.411649 (+ 401us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.411658 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.411690 (+ 32us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.411923 (+ 233us) tablet.cc:701] Row locks acquired
0505 07:54:09.411927 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:09.411998 (+ 71us) write_op.cc:270] Start()
0505 07:54:09.412055 (+ 57us) write_op.cc:276] Timestamp: P: 1777967649411950 usec, L: 0
0505 07:54:09.412060 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.412328 (+ 268us) log.cc:844] Serialized 725 byte log entry
0505 07:54:09.473135 (+ 60807us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.473494 (+ 359us) write_op.cc:301] APPLY: starting
0505 07:54:09.473511 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.473580 (+ 69us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.473640 (+ 60us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.473747 (+ 107us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.473791 (+ 44us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:09.473800 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:09.473881 (+ 81us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.473945 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.473983 (+ 38us) write_op.cc:454] Released schema lock
0505 07:54:09.473991 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":296,"apply.run_cpu_time_us":571,"apply.run_wall_time_us":571,"num_ops":2,"prepare.queue_time_us":130027,"prepare.run_cpu_time_us":955,"prepare.run_wall_time_us":1697,"replication_time_us":61036}]]}
I20260505 07:54:09.632102 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d86b8b4dadf4fd2b6409a8cb70e313e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.632398 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79a1da502604d05a62e27f575b8b8f4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:09.636721 718 leader_election.cc:343] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 3d83e9d4243e409794ffe8a342bce6b3
W20260505 07:54:09.667191 854 leader_election.cc:343] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:09.667366 854 leader_election.cc:343] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.642488 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 7219f9896fd94187ae714635077110af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.642855 745 tablet_service.cc:1558] Processing DeleteTablet for tablet 25072055f5534078adf5b5025614f951 with delete_type TABLET_DATA_DELETED (replaced by tablet 52e2db0240e04d6d8e948ebaedcc7d24) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.643193 748 tablet_service.cc:1558] Processing DeleteTablet for tablet e6a5f823db6143319d743d363a77ea47 with delete_type TABLET_DATA_DELETED (replaced by tablet 90612f0afb4949f491e625c61e40d247) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.643500 750 tablet_service.cc:1558] Processing DeleteTablet for tablet c1a61c1d03d544c8aa01537ead97f83c with delete_type TABLET_DATA_DELETED (replaced by tablet 080b2984fcf2478a8def6540da63554d) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.643844 743 tablet_service.cc:1558] Processing DeleteTablet for tablet d77ff90ec2ff4b6eb1795d149459c23a with delete_type TABLET_DATA_DELETED (replaced by tablet e5eaa91ecc1f43d18b313b6be4cc211e) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.644145 761 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f1feb0387bc4afead72346398dbe9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.644470 744 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a79578757824bfc9df129e3475e7af3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.644634 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 4662d4a3d88d4c2883a35048d6e11eb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.644913 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a60c015136246a3b02f70254c0bb67f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.645048 749 tablet_service.cc:1558] Processing DeleteTablet for tablet 9044763bed1a41739ecce4def4cca1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.645182 760 tablet_service.cc:1558] Processing DeleteTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e with delete_type TABLET_DATA_DELETED (replaced by tablet dd3fc8a9d9f04636b1f31bb88411c1da) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.645288 751 tablet_service.cc:1558] Processing DeleteTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.645391 757 tablet_service.cc:1558] Processing DeleteTablet for tablet bf76e868bbcb44b0b1887c84dd27f093 with delete_type TABLET_DATA_DELETED (replaced by tablet c837877946604ef29f95dac9dd449bf3) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.650357 1176 tablet_bootstrap.cc:492] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:09.669055 1176 ts_tablet_manager.cc:1403] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.061s user 0.004s sys 0.000s
I20260505 07:54:09.670006 1176 raft_consensus.cc:359] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.670264 1176 raft_consensus.cc:385] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.670372 1176 raft_consensus.cc:740] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.670850 1176 consensus_queue.cc:260] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.671554 1176 ts_tablet_manager.cc:1434] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:09.671978 1176 tablet_bootstrap.cc:492] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:09.673964 1176 tablet_bootstrap.cc:654] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.673979 2268 raft_consensus.cc:2272] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.674619 2268 ts_tablet_manager.cc:1916] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:09.677647 1172 tablet_bootstrap.cc:492] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
W20260505 07:54:09.641526 988 leader_election.cc:343] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 32c008bd99224a7595cecc607e5496a9
I20260505 07:54:09.677770 1172 ts_tablet_manager.cc:1403] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.003s
I20260505 07:54:09.677820 988 leader_election.cc:304] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.678076 2407 raft_consensus.cc:2749] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.678566 2407 consensus_queue.cc:1048] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.678587 1172 raft_consensus.cc:359] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.678732 1172 raft_consensus.cc:385] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.678786 1172 raft_consensus.cc:740] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.674149 2447 log.cc:1199] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/a9494c3b6a99487596a030856b371b68
I20260505 07:54:09.678995 1172 consensus_queue.cc:260] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.679489 1172 ts_tablet_manager.cc:1434] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260505 07:54:09.679889 988 leader_election.cc:343] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.680389 2447 ts_tablet_manager.cc:1950] T a9494c3b6a99487596a030856b371b68 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:09.680493 2407 consensus_queue.cc:1048] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.681089 2447 tablet_replica.cc:333] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:09.681267 2447 raft_consensus.cc:2243] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:09.681412 2447 raft_consensus.cc:2272] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:09.681638 988 leader_election.cc:343] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.682309 2447 ts_tablet_manager.cc:1916] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:09.693600 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e278b86e7b5d444ebad8d0f6921d4c62 (table default.Table_633da5b39da34f5c88163c9936fd0212 [id=d53c653124364ea5acc357977c9db864]) successfully deleted
I20260505 07:54:09.693892 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 05351965bdba4ccca8d13223cd505948 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:09.661978 1353 rpcz_store.cc:269] 0505 07:54:07.565637 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.565697 (+ 60us) service_pool.cc:224] Handling call
0505 07:54:09.661716 (+2096019us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:08.398686 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:08.398806 (+ 120us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:08.398811 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:08.398814 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:08.398938 (+ 124us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:08.398956 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:08.398960 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:08.399051 (+ 91us) tablet.cc:701] Row locks acquired
0505 07:54:08.399054 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:08.399094 (+ 40us) write_op.cc:270] Start()
0505 07:54:08.399128 (+ 34us) write_op.cc:276] Timestamp: P: 1777967648399052 usec, L: 0
0505 07:54:08.399135 (+ 7us) op_driver.cc:348] REPLICATION: starting
0505 07:54:08.399350 (+ 215us) log.cc:844] Serialized 901 byte log entry
0505 07:54:08.403153 (+ 3803us) op_driver.cc:464] REPLICATION: finished
0505 07:54:08.403279 (+ 126us) write_op.cc:301] APPLY: starting
0505 07:54:08.403294 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:08.403344 (+ 50us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:08.403375 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:08.403451 (+ 76us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:08.403492 (+ 41us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:08.403499 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:08.403568 (+ 69us) log.cc:844] Serialized 29 byte log entry
0505 07:54:08.403655 (+ 87us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:08.403676 (+ 21us) write_op.cc:454] Released schema lock
0505 07:54:08.403685 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:08.403758 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:08.403761 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
0505 07:54:08.440536 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:08.440667 (+ 131us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:08.440672 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:08.440674 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:08.440759 (+ 85us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:08.440779 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:08.440783 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:08.440828 (+ 45us) tablet.cc:701] Row locks acquired
0505 07:54:08.440831 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:08.440858 (+ 27us) write_op.cc:270] Start()
0505 07:54:08.440881 (+ 23us) write_op.cc:276] Timestamp: P: 1777967648440813 usec, L: 0
0505 07:54:08.440885 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:08.441023 (+ 138us) log.cc:844] Serialized 356 byte log entry
0505 07:54:08.448869 (+ 7846us) op_driver.cc:464] REPLICATION: finished
0505 07:54:08.449202 (+ 333us) write_op.cc:301] APPLY: starting
0505 07:54:08.449236 (+ 34us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:08.449269 (+ 33us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:08.449302 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:08.449358 (+ 56us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:08.449389 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:08.449400 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:08.449480 (+ 80us) log.cc:844] Serialized 23 byte log entry
0505 07:54:08.449534 (+ 54us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:08.449550 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:08.449557 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:08.449636 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:08.449642 (+ 6us) write_op.cc:454] Released schema lock
Related trace 'op':
0505 07:54:09.281190 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.281332 (+ 142us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.281339 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:09.281341 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:09.281475 (+ 134us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.281491 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.281495 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:09.281557 (+ 62us) tablet.cc:701] Row locks acquired
0505 07:54:09.281559 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:09.281595 (+ 36us) write_op.cc:270] Start()
0505 07:54:09.281641 (+ 46us) write_op.cc:276] Timestamp: P: 1777967649281561 usec, L: 0
0505 07:54:09.281645 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.281825 (+ 180us) log.cc:844] Serialized 356 byte log entry
0505 07:54:09.313685 (+ 31860us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.322450 (+ 8765us) write_op.cc:301] APPLY: starting
0505 07:54:09.322472 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.322524 (+ 52us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.322570 (+ 46us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.322636 (+ 66us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.322679 (+ 43us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:09.322689 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:09.322824 (+ 135us) log.cc:844] Serialized 23 byte log entry
0505 07:54:09.323673 (+ 849us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.323696 (+ 23us) write_op.cc:454] Released schema lock
0505 07:54:09.323706 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.323800 (+ 94us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.323805 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":174510,"reported_tablets":233,"child_traces":[["op",{"apply.queue_time_us":77,"apply.run_cpu_time_us":473,"apply.run_wall_time_us":472,"num_ops":2,"prepare.queue_time_us":4178,"prepare.run_cpu_time_us":780,"prepare.run_wall_time_us":780,"raft.queue_time_us":422,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":243,"replication_time_us":3984}],["op",{"apply.queue_time_us":280,"apply.run_cpu_time_us":425,"apply.run_wall_time_us":425,"mutex_wait_us":2074,"num_ops":1,"prepare.queue_time_us":2116,"prepare.run_cpu_time_us":553,"prepare.run_wall_time_us":553,"replication_time_us":7945}],["op",{"apply.queue_time_us":8684,"apply.run_cpu_time_us":583,"apply.run_wall_time_us":1344,"num_ops":1,"prepare.queue_time_us":72474,"prepare.run_cpu_time_us":760,"prepare.run_wall_time_us":1070,"raft.queue_time_us":3437,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":268,"replication_time_us":32005,"thread_start_us":74903,"threads_started":2}]]}
W20260505 07:54:09.694274 988 leader_election.cc:343] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 4ba2cc218300472cb5fd6dc8cfb7811c
W20260505 07:54:09.695715 988 consensus_peers.cc:597] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:09.694620 2307 consensus_queue.cc:1048] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:09.694578 991 leader_election.cc:304] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.696516 2307 raft_consensus.cc:2804] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:09.696627 2307 raft_consensus.cc:697] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:09.696846 2307 consensus_queue.cc:237] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:09.704717 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3], current schema version=2): Not found: failed to reset TS proxy: No leader found: Tablet bf0ce3e6c00541569d15c1b6ef347579 consensus state has no leader
I20260505 07:54:09.706997 1348 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 118) took 1994 ms. Trace:
I20260505 07:54:09.706521 1353 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_3ffe224aa51546ed94df1af5344e3d0a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:09.709321 2505 raft_consensus.cc:493] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:09.709986 1351 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.1:36755 (request call id 141) took 3861 ms (client timeout 1000 ms). Trace:
W20260505 07:54:09.710155 1351 rpcz_store.cc:269] 0505 07:54:05.844681 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:05.875898 (+ 31217us) service_pool.cc:224] Handling call
0505 07:54:05.904205 (+ 28307us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d1358 00007fd65a467ca6 00007fd65a4679d5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:09.706454 (+3802249us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:06.503273 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:06.503416 (+ 143us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:06.503423 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:06.503427 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:06.503544 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:06.503555 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:06.503562 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:06.503636 (+ 74us) tablet.cc:701] Row locks acquired
0505 07:54:06.503639 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:06.503666 (+ 27us) write_op.cc:270] Start()
0505 07:54:06.503693 (+ 27us) write_op.cc:276] Timestamp: P: 1777967646503608 usec, L: 0
0505 07:54:06.503698 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:06.617440 (+113742us) log.cc:844] Serialized 527 byte log entry
0505 07:54:06.617630 (+ 190us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65bae8f22 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac
0505 07:54:06.638753 (+ 21123us) op_driver.cc:464] REPLICATION: finished
0505 07:54:06.639032 (+ 279us) write_op.cc:301] APPLY: starting
0505 07:54:06.639044 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:06.639076 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:06.639080 (+ 4us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:06.639129 (+ 49us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:06.639163 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:06.639169 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:06.639243 (+ 74us) log.cc:844] Serialized 23 byte log entry
0505 07:54:06.639429 (+ 186us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:06.639443 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:06.639448 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:06.639506 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:06.639510 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22709,"reported_tablets":208,"spinlock_wait_cycles":40423168,"child_traces":[["op",{"apply.queue_time_us":203,"apply.run_cpu_time_us":342,"apply.run_wall_time_us":474,"num_ops":1,"prepare.queue_time_us":4602,"prepare.run_cpu_time_us":2781,"prepare.run_wall_time_us":114428,"raft.queue_time_us":2684,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":241,"replication_time_us":135028,"spinlock_wait_cycles":261098240}]]}
I20260505 07:54:09.711323 1351 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_829cc346ac834637ab2e2198944a8550" } new_table_name: "default.Table_b85a2b107c88498e84217252ca6cb947" modify_external_catalogs: true
I20260505 07:54:09.709892 2505 raft_consensus.cc:515] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.712754 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e99f96a753047348371bee5c022684e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.713045 1349 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 117) took 1994 ms. Trace:
W20260505 07:54:09.713233 854 leader_election.cc:343] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.713146 1349 rpcz_store.cc:276] 0505 07:54:07.712134 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.448909 (+1736775us) service_pool.cc:224] Handling call
0505 07:54:09.480393 (+ 31484us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d6b13 00007fd65a46a7c6 00007fd65a46a4f5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:09.481293 (+ 900us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:09.481864 (+ 571us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:09.509269 (+ 27405us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:09.706883 (+197614us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:09.706946 (+ 63us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.483633 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.483769 (+ 136us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.483777 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:09.483780 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:09.483891 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.483917 (+ 26us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.483927 (+ 10us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.484019 (+ 92us) tablet.cc:701] Row locks acquired
0505 07:54:09.484023 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:09.484063 (+ 40us) write_op.cc:270] Start()
0505 07:54:09.484099 (+ 36us) write_op.cc:276] Timestamp: P: 1777967649484008 usec, L: 0
0505 07:54:09.484103 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.484508 (+ 405us) log.cc:844] Serialized 433 byte log entry
0505 07:54:09.507322 (+ 22814us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.507506 (+ 184us) write_op.cc:301] APPLY: starting
0505 07:54:09.507521 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.507570 (+ 49us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.507615 (+ 45us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.507737 (+ 122us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.507768 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:09.507776 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:09.507840 (+ 64us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.507897 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.507913 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:09.507919 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.507983 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.507986 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":71592192,"child_traces":[["op",{"apply.queue_time_us":141,"apply.run_cpu_time_us":466,"apply.run_wall_time_us":465,"num_ops":2,"prepare.queue_time_us":61,"prepare.run_cpu_time_us":819,"prepare.run_wall_time_us":2741,"replication_time_us":23149}]]}
I20260505 07:54:09.714216 1349 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7af623f581af498b86c8c3b1adbc6b80" } modify_external_catalogs: true
I20260505 07:54:09.714437 1349 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7af623f581af498b86c8c3b1adbc6b80" } modify_external_catalogs: true
I20260505 07:54:09.715008 1348 rpcz_store.cc:276] 0505 07:54:07.712204 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.478518 (+1766314us) service_pool.cc:224] Handling call
0505 07:54:09.483872 (+ 5354us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d6b13 00007fd65a46a7c6 00007fd65a46a4f5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:09.484665 (+ 793us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:09.485119 (+ 454us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:09.509154 (+ 24035us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:09.706725 (+197571us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:09.706830 (+ 105us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.486427 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.486886 (+ 459us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.486896 (+ 10us) write_op.cc:435] Acquired schema lock
0505 07:54:09.486899 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:09.487285 (+ 386us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.487297 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.487304 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.487402 (+ 98us) tablet.cc:701] Row locks acquired
0505 07:54:09.487406 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:09.487439 (+ 33us) write_op.cc:270] Start()
0505 07:54:09.487471 (+ 32us) write_op.cc:276] Timestamp: P: 1777967649487386 usec, L: 0
0505 07:54:09.487476 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.487748 (+ 272us) log.cc:844] Serialized 433 byte log entry
0505 07:54:09.507424 (+ 19676us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.508045 (+ 621us) write_op.cc:301] APPLY: starting
0505 07:54:09.508054 (+ 9us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.508083 (+ 29us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.508102 (+ 19us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.508196 (+ 94us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.508220 (+ 24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:09.508226 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:09.508275 (+ 49us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.508310 (+ 35us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.508324 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:09.508329 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.508385 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.508388 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":11577600,"child_traces":[["op",{"apply.queue_time_us":592,"apply.run_cpu_time_us":333,"apply.run_wall_time_us":333,"num_ops":2,"prepare.queue_time_us":845,"prepare.run_cpu_time_us":813,"prepare.run_wall_time_us":1793,"replication_time_us":19931}]]}
I20260505 07:54:09.716410 2505 leader_election.cc:290] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.716430 1348 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6cb216f92da74a78acd72a6132b94c1a" } modify_external_catalogs: true
I20260505 07:54:09.716596 1348 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6cb216f92da74a78acd72a6132b94c1a" } modify_external_catalogs: true
W20260505 07:54:09.717221 1353 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ffe224aa51546ed94df1af5344e3d0a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:09.717519 1172 tablet_bootstrap.cc:492] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.718452 2505 raft_consensus.cc:493] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.718659 2505 raft_consensus.cc:515] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.719515 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9b7c6e9463342a5a73af12da4447f89" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:09.720099 854 leader_election.cc:343] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.721652 1172 tablet_bootstrap.cc:654] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:09.721397 1357 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.3:45737 (request call id 156) took 3148 ms (client timeout 1000 ms). Trace:
W20260505 07:54:09.722750 1358 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.3:45737 (request call id 157) took 1839 ms (client timeout 1000 ms). Trace:
W20260505 07:54:09.722941 1358 rpcz_store.cc:269] 0505 07:54:07.883231 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.891755 (+ 8524us) service_pool.cc:224] Handling call
0505 07:54:09.722690 (+1830935us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:08.599514 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:08.599673 (+ 159us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:08.599679 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:08.599681 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:08.599863 (+ 182us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:08.599877 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:08.599883 (+ 6us) tablet.cc:685] Acquiring locks for 3 operations
0505 07:54:08.599971 (+ 88us) tablet.cc:701] Row locks acquired
0505 07:54:08.599974 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:08.600007 (+ 33us) write_op.cc:270] Start()
0505 07:54:08.600038 (+ 31us) write_op.cc:276] Timestamp: P: 1777967648599944 usec, L: 0
0505 07:54:08.600041 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:08.600237 (+ 196us) log.cc:844] Serialized 1275 byte log entry
0505 07:54:08.603211 (+ 2974us) op_driver.cc:464] REPLICATION: finished
0505 07:54:08.603320 (+ 109us) write_op.cc:301] APPLY: starting
0505 07:54:08.603333 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:08.603389 (+ 56us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:08.603412 (+ 23us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:08.603481 (+ 69us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:08.603513 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0505 07:54:08.603519 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:08.603635 (+ 116us) log.cc:844] Serialized 35 byte log entry
0505 07:54:08.603692 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:08.603710 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:08.603716 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:08.603777 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:08.603780 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":51551,"reported_tablets":239,"child_traces":[["op",{"apply.queue_time_us":67,"apply.run_cpu_time_us":449,"apply.run_wall_time_us":449,"num_ops":3,"prepare.queue_time_us":1432,"prepare.run_cpu_time_us":807,"prepare.run_wall_time_us":819,"raft.queue_time_us":356,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":215,"replication_time_us":3141}]]}
I20260505 07:54:09.725256 1358 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_55fc26564d414d72992d4651acdbc6bb" } new_table_name: "default.Table_db06b2a6ada74addb6f217ab26d73870" modify_external_catalogs: true
I20260505 07:54:09.727803 1353 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 119) took 1960 ms. Trace:
I20260505 07:54:09.727991 1353 rpcz_store.cc:276] 0505 07:54:07.766934 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.694499 (+1927565us) service_pool.cc:224] Handling call
0505 07:54:09.717390 (+ 22891us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:09.717920 (+ 530us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:09.727524 (+ 9604us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:09.727664 (+ 140us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:09.727757 (+ 93us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.718842 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.718943 (+ 101us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.718950 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:09.718954 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:09.719061 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.719072 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.719080 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.719164 (+ 84us) tablet.cc:701] Row locks acquired
0505 07:54:09.719168 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:09.719202 (+ 34us) write_op.cc:270] Start()
0505 07:54:09.719232 (+ 30us) write_op.cc:276] Timestamp: P: 1777967649719178 usec, L: 0
0505 07:54:09.719238 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.719429 (+ 191us) log.cc:844] Serialized 433 byte log entry
0505 07:54:09.723376 (+ 3947us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.723475 (+ 99us) write_op.cc:301] APPLY: starting
0505 07:54:09.723487 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.723520 (+ 33us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.723549 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.723669 (+ 120us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.723696 (+ 27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:09.723703 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:09.723759 (+ 56us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.723806 (+ 47us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.723833 (+ 27us) write_op.cc:454] Released schema lock
0505 07:54:09.723838 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.723905 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.723912 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":59,"apply.run_cpu_time_us":419,"apply.run_wall_time_us":418,"num_ops":2,"prepare.queue_time_us":246,"prepare.run_cpu_time_us":659,"prepare.run_wall_time_us":659,"replication_time_us":4112}]]}
I20260505 07:54:09.729218 1353 master_service.cc:954] ReplaceTablet: received request to replace tablet f121fdd001ec4ea38ccd24a6afd9e51c from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:09.731195 1353 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 124) took 1950 ms. Trace:
I20260505 07:54:09.731371 1353 rpcz_store.cc:276] 0505 07:54:07.780927 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.729147 (+1948220us) service_pool.cc:224] Handling call
0505 07:54:09.731128 (+ 1981us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:09.732014 1353 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_4d8d7c4d5cc546a184cd68a8c7979589"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:09.733310 1353 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4d8d7c4d5cc546a184cd68a8c7979589 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:09.735793 717 leader_election.cc:343] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 3d83e9d4243e409794ffe8a342bce6b3
I20260505 07:54:09.745543 2505 leader_election.cc:290] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.746383 717 leader_election.cc:304] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
W20260505 07:54:09.749686 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 53991.1us
I20260505 07:54:09.750101 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79a1da502604d05a62e27f575b8b8f4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.750686 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d86b8b4dadf4fd2b6409a8cb70e313e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.752995 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e99f96a753047348371bee5c022684e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.753909 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9b7c6e9463342a5a73af12da4447f89" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:09.722607 1357 rpcz_store.cc:269] 0505 07:54:06.572768 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.573432 (+ 664us) service_pool.cc:224] Handling call
0505 07:54:07.585827 (+1012395us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.585836 (+ 9us) write_op.cc:454] Released schema lock
0505 07:54:09.721347 (+2135511us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:07.532446 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:07.532558 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:07.532566 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:07.532570 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:07.532679 (+ 109us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:07.532695 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:07.532701 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:07.532764 (+ 63us) tablet.cc:701] Row locks acquired
0505 07:54:07.532768 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:07.532816 (+ 48us) write_op.cc:270] Start()
0505 07:54:07.532849 (+ 33us) write_op.cc:276] Timestamp: P: 1777967647532762 usec, L: 0
0505 07:54:07.532852 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:07.533035 (+ 183us) log.cc:844] Serialized 527 byte log entry
0505 07:54:07.535862 (+ 2827us) op_driver.cc:464] REPLICATION: finished
0505 07:54:07.535982 (+ 120us) write_op.cc:301] APPLY: starting
0505 07:54:07.535999 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:07.536039 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:07.536075 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:07.536129 (+ 54us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:07.536163 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:07.536173 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:07.536251 (+ 78us) log.cc:844] Serialized 23 byte log entry
0505 07:54:07.536310 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:07.536343 (+ 33us) write_op.cc:454] Released schema lock
0505 07:54:07.536350 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":157866,"reported_tablets":206,"child_traces":[["op",{"apply.queue_time_us":69,"apply.run_cpu_time_us":415,"apply.run_wall_time_us":415,"num_ops":1,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":712,"prepare.run_wall_time_us":1309,"raft.queue_time_us":36,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":230,"replication_time_us":2979}]]}
I20260505 07:54:09.756208 1357 master_service.cc:954] ReplaceTablet: received request to replace tablet 62d4bc3154704ba2bfde83bca10f812d from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:09.756633 1357 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 126) took 1784 ms. Trace:
I20260505 07:54:09.756737 1357 rpcz_store.cc:276] 0505 07:54:07.971964 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.756087 (+1784123us) service_pool.cc:224] Handling call
0505 07:54:09.756593 (+ 506us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:09.756975 1357 master_service.cc:954] ReplaceTablet: received request to replace tablet 6fe19306f7df49fdb771d21f6560e697 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:09.762953 1181 tablet_bootstrap.cc:492] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:09.763208 1181 ts_tablet_manager.cc:1403] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.189s user 0.003s sys 0.002s
I20260505 07:54:09.763844 2089 raft_consensus.cc:2749] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.764083 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 99009.5us
I20260505 07:54:09.765005 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet a9494c3b6a99487596a030856b371b68 (table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]) successfully deleted
W20260505 07:54:09.765276 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 4f3e076892234c1494b1a5c13cbfd2e2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f3e076892234c1494b1a5c13cbfd2e2 already in progress: creating tablet
W20260505 07:54:09.765497 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet a9b7c6e9463342a5a73af12da4447f89 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9b7c6e9463342a5a73af12da4447f89 already in progress: creating tablet
I20260505 07:54:09.765944 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4bc3154704ba2bfde83bca10f812d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.766130 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 9959be9f83c2444282d012b61eef49b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.766281 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b57c1c618484d5d86a846fb982a2240 with delete_type TABLET_DATA_DELETED (replaced by tablet de1a667b14fe494c8ee4be0f47a340be) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.766433 760 tablet_service.cc:1558] Processing DeleteTablet for tablet feeafd614b3e4281a8e45b950e699b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.766590 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 0baf83b155494fddb3c25144608706f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.766752 760 tablet_service.cc:1558] Processing DeleteTablet for tablet f92c53c68a4b4e319e98b84128e8e6a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.766909 751 tablet_service.cc:1558] Processing DeleteTablet for tablet bc9ae6404dd64bd8bf4898768e94e4f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.767064 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 141665fcd59048bf926acd0287f1853e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.767218 751 tablet_service.cc:1558] Processing DeleteTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.769569 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:09.774453 1181 raft_consensus.cc:359] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.774730 1181 raft_consensus.cc:385] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.774845 1181 raft_consensus.cc:740] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.775094 1181 consensus_queue.cc:260] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.775764 1181 ts_tablet_manager.cc:1434] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260505 07:54:09.776073 1181 tablet_bootstrap.cc:492] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:09.777655 1181 tablet_bootstrap.cc:654] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.778182 1176 tablet_bootstrap.cc:492] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:09.780491 991 leader_election.cc:304] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.780857 1878 raft_consensus.cc:2804] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:09.780926 1878 raft_consensus.cc:493] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:09.780982 1878 raft_consensus.cc:3060] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:09.783370 1878 raft_consensus.cc:515] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.785578 2505 raft_consensus.cc:493] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.785718 2505 raft_consensus.cc:515] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.786360 2392 raft_consensus.cc:493] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.786495 2392 raft_consensus.cc:515] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.787024 2392 leader_election.cc:290] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.788159 1176 ts_tablet_manager.cc:1403] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.116s user 0.002s sys 0.002s
I20260505 07:54:09.788425 1878 leader_election.cc:290] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:09.789085 991 leader_election.cc:304] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.787828 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1602ccbae24544c4a6cf14ea073880b4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.789304 1878 raft_consensus.cc:2804] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:09.789381 1878 raft_consensus.cc:493] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:09.789446 1878 raft_consensus.cc:3060] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Advancing to term 2
W20260505 07:54:09.789743 854 leader_election.cc:343] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.787418 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b256d072933b4566b5587224274deb66" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.788296 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 4258df74ea0841449846cdf1f6e00b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:09.790330 854 leader_election.cc:343] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.790382 757 tablet_service.cc:1558] Processing DeleteTablet for tablet b5a0cbf9c3d74cea92920189da332c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.788456 2505 leader_election.cc:290] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.790930 1176 raft_consensus.cc:359] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.791133 1176 raft_consensus.cc:385] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.791234 1176 raft_consensus.cc:740] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.791458 1176 consensus_queue.cc:260] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.791805 1878 raft_consensus.cc:515] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:09.792124 757 tablet_service.cc:1558] Processing DeleteTablet for tablet d3d11bd19f074bbab101448b45426b7a with delete_type TABLET_DATA_DELETED (replaced by tablet 76939b7978ee4f7ea0ad29312de2a31a) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.792227 1176 ts_tablet_manager.cc:1434] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:09.792392 1878 leader_election.cc:290] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:09.792815 856 leader_election.cc:343] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.792971 856 leader_election.cc:304] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:09.793330 856 leader_election.cc:343] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.793452 856 leader_election.cc:304] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.793663 2392 raft_consensus.cc:2749] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.793774 2392 raft_consensus.cc:2749] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.794968 856 leader_election.cc:343] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.795114 856 leader_election.cc:304] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.795338 2392 raft_consensus.cc:2749] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.795334 2276 ts_tablet_manager.cc:1929] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:09.795711 2276 log.cc:1199] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/06904a54b1c5463da31af0bc00fb606f
I20260505 07:54:09.795976 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a73dbb979344805876efbafbf1b2985" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:09.795749 1172 tablet_bootstrap.cc:492] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.796249 1172 ts_tablet_manager.cc:1403] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.079s user 0.002s sys 0.002s
I20260505 07:54:09.796882 1176 tablet_bootstrap.cc:492] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:09.797309 2276 ts_tablet_manager.cc:1950] T 06904a54b1c5463da31af0bc00fb606f P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
W20260505 07:54:09.795547 856 leader_election.cc:343] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:09.797673 1318 catalog_manager.cc:5081] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): alter failed for tablet accb2dedc30e494ebd22c33594cee661 (table default.Table_fa6246cc3ccd4d7082becde923f88828 [id=860ab29ce1a44ad1b2480dbdb2b40000]): Aborted: Op aborted
I20260505 07:54:09.797849 856 leader_election.cc:304] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.798249 2392 raft_consensus.cc:2749] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.798496 915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ba2cc218300472cb5fd6dc8cfb7811c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:09.798652 915 raft_consensus.cc:3060] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:09.798606 1172 raft_consensus.cc:359] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.798741 1172 raft_consensus.cc:385] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.798794 1172 raft_consensus.cc:740] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.798997 1172 consensus_queue.cc:260] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.799610 1172 ts_tablet_manager.cc:1434] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:09.799860 1172 tablet_bootstrap.cc:492] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.800161 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a73dbb979344805876efbafbf1b2985" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:09.800318 911 raft_consensus.cc:3060] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:09.801864 1172 tablet_bootstrap.cc:654] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.802117 915 raft_consensus.cc:2468] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 2.
I20260505 07:54:09.802837 911 raft_consensus.cc:2468] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:09.806741 988 leader_election.cc:343] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 4ba2cc218300472cb5fd6dc8cfb7811c
W20260505 07:54:09.806986 988 leader_election.cc:343] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.807334 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f207a3ffbc94b01abd9e9d6837f5c81) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.807521 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 02412333633a4cdebe95a144fa6c363c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.807734 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 46274a44a06b4306b5dac43c66c85a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.807854 751 tablet_service.cc:1558] Processing DeleteTablet for tablet e29ba55a846548679cb3b86305d0d6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.807957 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 3075e87e078f4d2aa1d329d18902354b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.808060 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 27bacc97d7534284970f218cf68ac75b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.809094 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 05351965bdba4ccca8d13223cd505948 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.810127 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e08cf03ea0b495a94c6566ea6c8caad) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.813067 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.813730 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 32c008bd99224a7595cecc607e5496a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a9494c3b6a99487596a030856b371b68) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.814337 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 with delete_type TABLET_DATA_DELETED (replaced by tablet 66082a7a75e345639e7a268727922fbb) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.814689 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1602ccbae24544c4a6cf14ea073880b4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.815147 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet e82e9ae417784b7e83f4843b129d0580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.815505 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b256d072933b4566b5587224274deb66" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.816704 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.817314 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet bd581a6799a14a8dafb80d411e8936f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.818673 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.819092 2445 tablet.cc:1724] T 1001dd888ecd4166ba217506739f71e8 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:09.819720 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 93d878ae0ddb42319560f74e0a490321 with delete_type TABLET_DATA_DELETED (replaced by tablet bc9ae6404dd64bd8bf4898768e94e4f1) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:09.821985 856 leader_election.cc:343] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.822157 2268 ts_tablet_manager.cc:1929] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:09.822283 2268 log.cc:1199] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/accb2dedc30e494ebd22c33594cee661
I20260505 07:54:09.822626 2268 ts_tablet_manager.cc:1950] T accb2dedc30e494ebd22c33594cee661 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:09.823156 1181 tablet_bootstrap.cc:492] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:09.823244 856 leader_election.cc:304] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.823516 2392 raft_consensus.cc:2749] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.824039 856 leader_election.cc:343] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.824343 1181 ts_tablet_manager.cc:1403] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.048s user 0.003s sys 0.000s
I20260505 07:54:09.824513 856 leader_election.cc:304] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.825166 2392 raft_consensus.cc:2749] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.824136 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 63) took 2179 ms. Trace:
I20260505 07:54:09.825651 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a06f77255544d7dabb71a4f70ada809 with delete_type TABLET_DATA_DELETED (replaced by tablet 12841efcef6041d89fc6d7d9b90448c0) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.825856 2276 rpcz_store.cc:276] 0505 07:54:07.644685 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.644793 (+ 108us) service_pool.cc:224] Handling call
0505 07:54:09.639775 (+1994982us) ts_tablet_manager.cc:1151] Deleting tablet 06904a54b1c5463da31af0bc00fb606f
0505 07:54:09.639785 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.744081 (+104296us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.795257 (+ 51176us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.824118 (+ 28861us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1991544}
W20260505 07:54:09.826076 444 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.105.60:35401 timed out after 0.608s (SENT) [suppressed 1 similar messages]
I20260505 07:54:09.832149 2392 raft_consensus.cc:493] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.843024 2392 raft_consensus.cc:515] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.844137 2392 leader_election.cc:290] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.844696 2276 tablet_replica.cc:333] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:09.844987 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 06904a54b1c5463da31af0bc00fb606f (table default.Table_577ff393d2af447a915f4460ab7692f1 [id=96336409bde3400fb18cb06e0d84e99c]) successfully deleted
I20260505 07:54:09.845256 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e with delete_type TABLET_DATA_DELETED (replaced by tablet dd3fc8a9d9f04636b1f31bb88411c1da) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.846459 2276 raft_consensus.cc:2243] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:09.846736 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 42) took 2764 ms. Trace:
I20260505 07:54:09.846830 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet d77ff90ec2ff4b6eb1795d149459c23a with delete_type TABLET_DATA_DELETED (replaced by tablet e5eaa91ecc1f43d18b313b6be4cc211e) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.847448 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet c1a61c1d03d544c8aa01537ead97f83c with delete_type TABLET_DATA_DELETED (replaced by tablet 080b2984fcf2478a8def6540da63554d) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.847915 1181 raft_consensus.cc:359] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.848380 1181 raft_consensus.cc:385] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.848641 1181 raft_consensus.cc:740] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.852644 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66be7b6c33a342d6890e459d90d40583" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.847132 2268 rpcz_store.cc:276] 0505 07:54:07.082715 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.114443 (+ 31728us) service_pool.cc:224] Handling call
0505 07:54:09.594715 (+2480272us) ts_tablet_manager.cc:1151] Deleting tablet accb2dedc30e494ebd22c33594cee661
0505 07:54:09.594727 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.777483 (+182756us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.822101 (+ 44618us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.846717 (+ 24616us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22656,"tablet-delete.queue_time_us":2480166}
I20260505 07:54:09.859427 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 02716f62aa354d8f839266661b7f05ea with delete_type TABLET_DATA_DELETED (replaced by tablet 9f541eff399641dd81a0771186805493) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.847965 2276 raft_consensus.cc:2272] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.861164 2276 ts_tablet_manager.cc:1916] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:09.861567 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 05351965bdba4ccca8d13223cd505948 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.863822 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 976359601ad74b149696dc7f555b550c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.864550 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 with delete_type TABLET_DATA_DELETED (replaced by tablet 42360f8ee4ff47c2af8d287f792207de) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.867147 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e707143a4e64725aa66db05b24999a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.868818 771 raft_consensus.cc:3060] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:09.853644 856 leader_election.cc:343] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.869467 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 1852f7f4ff594639908b03730b68fe86 with delete_type TABLET_DATA_DELETED (replaced by tablet aab3850799254dda9716fd5bf9404159) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.855432 1181 consensus_queue.cc:260] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.870730 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 397d33e908af4a0bb8dde26d273be01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.871356 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.872128 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet a9494c3b6a99487596a030856b371b68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6fe19306f7df49fdb771d21f6560e697) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.873824 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66be7b6c33a342d6890e459d90d40583" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:09.874364 854 leader_election.cc:343] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.874469 854 leader_election.cc:304] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.874877 2392 raft_consensus.cc:2749] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.875916 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.877991 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet f1058fcb9fa6457a85489049ca1d9ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.878383 1357 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 6fe19306f7df49fdb771d21f6560e697
I20260505 07:54:09.878945 1357 catalog_manager.cc:6741] ReplaceTablet: tablet 6fe19306f7df49fdb771d21f6560e697 deleted and replaced by tablet 317144923d724f6fa96ad49809d25b46
I20260505 07:54:09.879092 1357 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 127) took 1854 ms. Trace:
I20260505 07:54:09.879521 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 6fe19306f7df49fdb771d21f6560e697 with delete_type TABLET_DATA_DELETED (replaced by tablet 317144923d724f6fa96ad49809d25b46) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:09.879191 1357 rpcz_store.cc:276] 0505 07:54:08.024341 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.756916 (+1732575us) service_pool.cc:224] Handling call
0505 07:54:09.879034 (+122118us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.795746 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.795866 (+ 120us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.795873 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:09.795876 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:09.795991 (+ 115us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.796008 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.796013 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.796098 (+ 85us) tablet.cc:701] Row locks acquired
0505 07:54:09.796102 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:09.796141 (+ 39us) write_op.cc:270] Start()
0505 07:54:09.796174 (+ 33us) write_op.cc:276] Timestamp: P: 1777967649796109 usec, L: 0
0505 07:54:09.796199 (+ 25us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.796399 (+ 200us) log.cc:844] Serialized 622 byte log entry
0505 07:54:09.817691 (+ 21292us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.817791 (+ 100us) write_op.cc:301] APPLY: starting
0505 07:54:09.817806 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.817875 (+ 69us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.817899 (+ 24us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.817998 (+ 99us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.818054 (+ 56us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:09.818061 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:09.818124 (+ 63us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.818179 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.818213 (+ 34us) write_op.cc:454] Released schema lock
0505 07:54:09.818219 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.818280 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.818283 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":58,"apply.run_cpu_time_us":480,"apply.run_wall_time_us":480,"num_ops":2,"prepare.queue_time_us":37592,"prepare.run_cpu_time_us":756,"prepare.run_wall_time_us":756,"raft.queue_time_us":414,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":224,"replication_time_us":21461}]]}
I20260505 07:54:09.879997 1357 master_service.cc:954] ReplaceTablet: received request to replace tablet 5683d7366de84653ace56f0edcd3818c from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:09.880554 899 tablet_service.cc:1511] Processing CreateTablet for tablet 19f8d177646146c2bda321745d12d170 (DEFAULT_TABLE table=default.Table_c00d8489b8f84ee3b9eb57d5f51514ba [id=99fa76ce2e0448158bc0b4b70a4009d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:09.880724 1353 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 125) took 1975 ms. Trace:
I20260505 07:54:09.881006 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19f8d177646146c2bda321745d12d170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:09.880829 1353 rpcz_store.cc:276] 0505 07:54:07.905302 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.731706 (+1826404us) service_pool.cc:224] Handling call
0505 07:54:09.733513 (+ 1807us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:09.735702 (+ 2189us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:09.835929 (+100227us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:09.880535 (+ 44606us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:09.880664 (+ 129us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.805282 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.805385 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.805390 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:09.805392 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:09.805451 (+ 59us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.805458 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.805461 (+ 3us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.805514 (+ 53us) tablet.cc:701] Row locks acquired
0505 07:54:09.805516 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:09.805538 (+ 22us) write_op.cc:270] Start()
0505 07:54:09.805571 (+ 33us) write_op.cc:276] Timestamp: P: 1777967649805509 usec, L: 0
0505 07:54:09.805575 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.806621 (+ 1046us) log.cc:844] Serialized 433 byte log entry
0505 07:54:09.828597 (+ 21976us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.829657 (+ 1060us) write_op.cc:301] APPLY: starting
0505 07:54:09.829665 (+ 8us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.829690 (+ 25us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.829715 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.829804 (+ 89us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.829836 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:09.829842 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:09.829892 (+ 50us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.829931 (+ 39us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.829944 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:09.829949 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1032,"apply.run_cpu_time_us":339,"apply.run_wall_time_us":338,"mutex_wait_us":9514,"num_ops":2,"prepare.queue_time_us":11069,"prepare.run_cpu_time_us":500,"prepare.run_wall_time_us":1393,"replication_time_us":23002,"threads_started":1}]]}
I20260505 07:54:09.886047 1176 tablet_bootstrap.cc:654] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.888772 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 02e06d8815034d91986cef12964860ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.889575 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 43) took 2806 ms. Trace:
I20260505 07:54:09.889669 2447 ts_tablet_manager.cc:1929] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:09.889822 2447 log.cc:1199] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/909306e1e5f34b7c8fbc0d86cafce531
I20260505 07:54:09.890218 2447 ts_tablet_manager.cc:1950] T 909306e1e5f34b7c8fbc0d86cafce531 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:09.889583 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.891119 1181 ts_tablet_manager.cc:1434] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.066s user 0.003s sys 0.000s
I20260505 07:54:09.892107 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 60772a9697ba417b879dc6025eeb8eec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.893042 1181 tablet_bootstrap.cc:492] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:09.895774 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.895957 2392 raft_consensus.cc:493] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.896108 2392 raft_consensus.cc:515] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.896478 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 09b46ff0101a4e529077b9e30492ace7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.896898 2444 raft_consensus.cc:493] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.897013 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet d896e290167b4d0081e8e8b831c8ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.897014 2444 raft_consensus.cc:515] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.897222 2392 leader_election.cc:290] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.897535 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet abaa596871bb4a82aefc317932e91d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.897544 2444 leader_election.cc:290] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.897914 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3075e87e078f4d2aa1d329d18902354b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:09.898487 854 leader_election.cc:343] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.898717 1181 tablet_bootstrap.cc:654] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.899629 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet e8bf49b787ef4de095448751464708a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:09.899986 1315 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet bf0ce3e6c00541569d15c1b6ef347579 consensus state has no leader
I20260505 07:54:09.900336 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.900897 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 27bacc97d7534284970f218cf68ac75b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.901475 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 8eb7eaff8837485a8ae3ccb44c60471d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.902015 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 78a6470720984761a8606abeafbef992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:09.902623 444 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.60:35401, user_credentials={real_user=slave}} blocked reactor thread for 76442.4us
I20260505 07:54:09.903741 1353 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_c67b8faa7954415aa740a42b0915aa34"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:09.903889 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 202a237ca580482c98803683a77d96d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:09.904302 1353 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c67b8faa7954415aa740a42b0915aa34 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:09.907776 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 003cd4472b424aacbba8c5f89900e228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.910706 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.911299 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet d622612bdc404727a27b83769bf39738 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.911686 757 tablet_service.cc:1511] Processing CreateTablet for tablet 19f8d177646146c2bda321745d12d170 (DEFAULT_TABLE table=default.Table_c00d8489b8f84ee3b9eb57d5f51514ba [id=99fa76ce2e0448158bc0b4b70a4009d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:09.911944 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f1feb0387bc4afead72346398dbe9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.912426 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 6fe19306f7df49fdb771d21f6560e697 with delete_type TABLET_DATA_DELETED (replaced by tablet 317144923d724f6fa96ad49809d25b46) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:09.912506 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 03f31efbc98c484c950c70d5e84d2136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.892524 2268 rpcz_store.cc:276] 0505 07:54:07.082786 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.114525 (+ 31739us) service_pool.cc:224] Handling call
0505 07:54:09.869193 (+2754668us) ts_tablet_manager.cc:1151] Deleting tablet a9b7c6e9463342a5a73af12da4447f89
0505 07:54:09.869720 (+ 527us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.889540 (+ 19820us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2745298}
I20260505 07:54:09.913033 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 323269e708964848a94c64e6a781a860 with delete_type TABLET_DATA_DELETED (replaced by tablet e010f23c4df647ab863e03c3cfed7898) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.915215 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 44) took 2832 ms. Trace:
I20260505 07:54:09.915302 2268 rpcz_store.cc:276] 0505 07:54:07.082913 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.114616 (+ 31703us) service_pool.cc:224] Handling call
0505 07:54:09.915106 (+2800490us) ts_tablet_manager.cc:1151] Deleting tablet ae829daad64545e687a9528e5c38b9ed
0505 07:54:09.915117 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.915199 (+ 82us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2800386}
I20260505 07:54:09.915668 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 45) took 2832 ms. Trace:
I20260505 07:54:09.915746 2268 rpcz_store.cc:276] 0505 07:54:07.082981 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.114704 (+ 31723us) service_pool.cc:224] Handling call
0505 07:54:09.915540 (+2800836us) ts_tablet_manager.cc:1151] Deleting tablet b12d2d2b866348d781209fa787da7d7e
0505 07:54:09.915558 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.915652 (+ 94us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2800690}
I20260505 07:54:09.915977 2268 tablet_replica.cc:333] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:09.916147 2268 raft_consensus.cc:2243] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:09.916290 2268 raft_consensus.cc:2272] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.916697 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 9959be9f83c2444282d012b61eef49b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.916814 2268 ts_tablet_manager.cc:1916] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:09.917785 1181 tablet_bootstrap.cc:492] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:09.917977 1181 ts_tablet_manager.cc:1403] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20260505 07:54:09.918243 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.918629 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a60c015136246a3b02f70254c0bb67f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.918948 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 9044763bed1a41739ecce4def4cca1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.923511 1181 raft_consensus.cc:359] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.926537 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a0cbf9c3d74cea92920189da332c1c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:09.927104 854 leader_election.cc:343] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.927917 1181 raft_consensus.cc:385] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.928040 1181 raft_consensus.cc:740] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.928169 2089 raft_consensus.cc:493] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.928293 2089 raft_consensus.cc:515] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.928288 1181 consensus_queue.cc:260] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:09.922173 988 consensus_peers.cc:597] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f88a5c0d0e2c436c8d8b6afc1f3c2edf. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:09.928849 2089 leader_election.cc:290] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.928831 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a0cbf9c3d74cea92920189da332c1c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.929354 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3075e87e078f4d2aa1d329d18902354b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:09.919245 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 12501fdf2cc5491cb1a06db490e70810 with delete_type TABLET_DATA_DELETED (replaced by tablet d622612bdc404727a27b83769bf39738) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.924310 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 06904a54b1c5463da31af0bc00fb606f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.924578 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet 26f3544e74d941d286a2a0317b941ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.925253 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet bf76e868bbcb44b0b1887c84dd27f093 with delete_type TABLET_DATA_DELETED (replaced by tablet c837877946604ef29f95dac9dd449bf3) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.925936 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.926491 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 17327cee69a24e0cb493ff55f948bf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.927070 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf with delete_type TABLET_DATA_DELETED (replaced by tablet 3a73dbb979344805876efbafbf1b2985) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.930318 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f3544e74d941d286a2a0317b941ab6" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:09.930390 718 leader_election.cc:343] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 26f3544e74d941d286a2a0317b941ab6
W20260505 07:54:09.930437 856 leader_election.cc:343] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.930624 856 leader_election.cc:304] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.930867 717 leader_election.cc:304] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:09.930946 2505 raft_consensus.cc:2749] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.931078 2089 raft_consensus.cc:2749] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:09.931504 856 leader_election.cc:343] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.931710 856 leader_election.cc:304] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.931959 2505 raft_consensus.cc:2749] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:09.937237 1353 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 129) took 1855 ms. Trace:
I20260505 07:54:09.937379 1353 rpcz_store.cc:276] 0505 07:54:08.081521 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.881856 (+1800335us) service_pool.cc:224] Handling call
0505 07:54:09.903672 (+ 21816us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d6b13 00007fd65a46a7c6 00007fd65a46a4f5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:09.904405 (+ 733us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:09.904899 (+ 494us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:09.936994 (+ 32095us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:09.937096 (+ 102us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:09.937186 (+ 90us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.923082 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.923184 (+ 102us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.923189 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:09.923191 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:09.923267 (+ 76us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.923278 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.923282 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.923345 (+ 63us) tablet.cc:701] Row locks acquired
0505 07:54:09.923347 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:09.923381 (+ 34us) write_op.cc:270] Start()
0505 07:54:09.923410 (+ 29us) write_op.cc:276] Timestamp: P: 1777967649923337 usec, L: 0
0505 07:54:09.923413 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.923632 (+ 219us) log.cc:844] Serialized 433 byte log entry
0505 07:54:09.926853 (+ 3221us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.936116 (+ 9263us) write_op.cc:301] APPLY: starting
0505 07:54:09.936132 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.936177 (+ 45us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.936229 (+ 52us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.936385 (+ 156us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.936417 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:09.936424 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:09.936501 (+ 77us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.936740 (+ 239us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.936758 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:09.936767 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.936851 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.936856 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":49506176,"child_traces":[["op",{"apply.queue_time_us":9173,"apply.run_cpu_time_us":565,"apply.run_wall_time_us":731,"num_ops":2,"prepare.queue_time_us":46,"prepare.run_cpu_time_us":660,"prepare.run_wall_time_us":661,"raft.queue_time_us":357,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":223,"replication_time_us":3413}]]}
I20260505 07:54:09.938263 2276 ts_tablet_manager.cc:1929] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:09.938783 1353 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_2eebf692e6ca4f6f8f7a1c82bbf85823"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:09.940232 1353 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2eebf692e6ca4f6f8f7a1c82bbf85823 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:09.943756 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet a7d8265c05b74b28959d0de41fad1fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.944087 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 with delete_type TABLET_DATA_DELETED (replaced by tablet d5083f98e8e14ccf99a19cdf2bc59018) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.944365 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.944679 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b57c1c618484d5d86a846fb982a2240 with delete_type TABLET_DATA_DELETED (replaced by tablet de1a667b14fe494c8ee4be0f47a340be) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.944974 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4bc3154704ba2bfde83bca10f812d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.945252 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 141665fcd59048bf926acd0287f1853e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.945603 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet feeafd614b3e4281a8e45b950e699b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.945947 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet b5a0cbf9c3d74cea92920189da332c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.946286 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet f92c53c68a4b4e319e98b84128e8e6a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.946619 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet bc9ae6404dd64bd8bf4898768e94e4f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.946986 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 0baf83b155494fddb3c25144608706f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.947315 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.948031 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 6fe19306f7df49fdb771d21f6560e697 with delete_type TABLET_DATA_DELETED (replaced by tablet 317144923d724f6fa96ad49809d25b46) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:09.948523 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 19f8d177646146c2bda321745d12d170 (DEFAULT_TABLE table=default.Table_c00d8489b8f84ee3b9eb57d5f51514ba [id=99fa76ce2e0448158bc0b4b70a4009d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:09.949177 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet accb2dedc30e494ebd22c33594cee661 (table default.Table_fa6246cc3ccd4d7082becde923f88828 [id=860ab29ce1a44ad1b2480dbdb2b40000]) successfully deleted
W20260505 07:54:09.949767 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet a9b7c6e9463342a5a73af12da4447f89 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a9b7c6e9463342a5a73af12da4447f89 already in progress: creating tablet
W20260505 07:54:09.949980 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet ae829daad64545e687a9528e5c38b9ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae829daad64545e687a9528e5c38b9ed already in progress: creating tablet
W20260505 07:54:09.950183 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet b12d2d2b866348d781209fa787da7d7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b12d2d2b866348d781209fa787da7d7e already in progress: creating tablet
I20260505 07:54:09.950500 1181 ts_tablet_manager.cc:1434] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.028s user 0.001s sys 0.000s
W20260505 07:54:09.950485 1318 catalog_manager.cc:5075] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): alter failed for tablet accb2dedc30e494ebd22c33594cee661 (table default.Table_fa6246cc3ccd4d7082becde923f88828 [id=860ab29ce1a44ad1b2480dbdb2b40000]),no further retry: Not found: Tablet not found: accb2dedc30e494ebd22c33594cee661
I20260505 07:54:09.950760 1181 tablet_bootstrap.cc:492] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:09.952688 1181 tablet_bootstrap.cc:654] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.956856 2405 raft_consensus.cc:493] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.956969 2405 raft_consensus.cc:515] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:09.957233 988 consensus_peers.cc:597] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:09.957566 2405 leader_election.cc:290] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:09.961328 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19f8d177646146c2bda321745d12d170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:09.962826 2445 tablet.cc:1724] T 62bf3a6bc0ac42a99a4181f820e3ca87 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:09.967514 1176 tablet_bootstrap.cc:492] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:09.968329 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f121fdd001ec4ea38ccd24a6afd9e51c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:09.968832 988 leader_election.cc:343] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.969740 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 909306e1e5f34b7c8fbc0d86cafce531 (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]) successfully deleted
W20260505 07:54:09.970003 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e82e9ae417784b7e83f4843b129d0580 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e82e9ae417784b7e83f4843b129d0580 already in progress: creating tablet
W20260505 07:54:09.970255 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d86b8b4dadf4fd2b6409a8cb70e313e already in progress: creating tablet
I20260505 07:54:09.969410 2447 tablet_replica.cc:333] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:09.969324 1176 ts_tablet_manager.cc:1403] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.177s user 0.003s sys 0.001s
I20260505 07:54:09.970964 2447 raft_consensus.cc:2243] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:09.974337 2447 raft_consensus.cc:2272] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.978979 1176 raft_consensus.cc:359] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.948491 2505 raft_consensus.cc:493] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.956051 1172 tablet_bootstrap.cc:492] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:09.976053 2444 raft_consensus.cc:493] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:09.980315 1172 ts_tablet_manager.cc:1403] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.181s user 0.005s sys 0.002s
I20260505 07:54:09.980264 2505 raft_consensus.cc:515] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.980288 2444 raft_consensus.cc:515] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.980898 2444 leader_election.cc:290] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.980919 2505 leader_election.cc:290] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:09.980569 1176 raft_consensus.cc:385] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.981679 2276 log.cc:1199] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/a7d8265c05b74b28959d0de41fad1fde
I20260505 07:54:09.981403 1176 raft_consensus.cc:740] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.982040 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad8a79245ad47738d079752e51aed2b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.982198 2276 ts_tablet_manager.cc:1950] T a7d8265c05b74b28959d0de41fad1fde P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:09.982486 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 68) took 2333 ms. Trace:
W20260505 07:54:09.982529 718 consensus_peers.cc:597] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 06904a54b1c5463da31af0bc00fb606f. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:09.982623 2276 rpcz_store.cc:276] 0505 07:54:07.648729 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.648966 (+ 237us) service_pool.cc:224] Handling call
0505 07:54:09.844641 (+2195675us) ts_tablet_manager.cc:1151] Deleting tablet a7d8265c05b74b28959d0de41fad1fde
0505 07:54:09.844653 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.905786 (+ 61133us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.938219 (+ 32433us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:09.982471 (+ 44252us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2195509}
W20260505 07:54:09.982836 854 leader_election.cc:343] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.982987 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet a7d8265c05b74b28959d0de41fad1fde (table default.Table_4e8ee4d80a174660827b921c9f211ac4 [id=ce8bd78012db468e8a358f51c308a86e]) successfully deleted
I20260505 07:54:09.983362 2276 tablet_replica.cc:333] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:09.983523 1172 raft_consensus.cc:359] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.984078 1176 consensus_queue.cc:260] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.985941 2447 ts_tablet_manager.cc:1916] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:09.988590 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19f8d177646146c2bda321745d12d170. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:09.988806 991 leader_election.cc:304] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:09.989171 2438 raft_consensus.cc:2804] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:09.989305 2438 raft_consensus.cc:697] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:09.989521 2438 consensus_queue.cc:237] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:09.989985 718 consensus_peers.cc:597] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:09.988970 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:09.990262 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37765950e67d466683551f67a35e9bdb" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:09.990353 1176 ts_tablet_manager.cc:1434] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.019s user 0.003s sys 0.000s
I20260505 07:54:09.991039 1172 raft_consensus.cc:385] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:09.991199 1172 raft_consensus.cc:740] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:09.991415 1172 consensus_queue.cc:260] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:09.992067 1172 ts_tablet_manager.cc:1434] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
I20260505 07:54:09.983631 2276 raft_consensus.cc:2243] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:09.993134 1176 tablet_bootstrap.cc:492] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:54:09.990890 854 leader_election.cc:343] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:09.992406 1172 tablet_bootstrap.cc:492] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:09.993304 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:09.995327 1172 tablet_bootstrap.cc:654] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:09.996238 2276 raft_consensus.cc:2272] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:09.996786 2276 ts_tablet_manager.cc:1916] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:09.997704 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:10.002597 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:10.006824 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad8a79245ad47738d079752e51aed2b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.007226 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37765950e67d466683551f67a35e9bdb" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:10.007371 856 leader_election.cc:343] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.007242 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:10.007630 856 leader_election.cc:304] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.007856 2505 raft_consensus.cc:2749] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:10.008024 856 leader_election.cc:343] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.008119 856 leader_election.cc:304] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.008306 2505 raft_consensus.cc:2749] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.008635 991 leader_election.cc:304] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.009033 2405 raft_consensus.cc:2804] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:10.009186 2405 raft_consensus.cc:697] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:10.011246 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:10.013139 1176 tablet_bootstrap.cc:654] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.013875 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f121fdd001ec4ea38ccd24a6afd9e51c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.014071 916 raft_consensus.cc:2468] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:10.013293 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:09.994218 1353 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 130) took 1896 ms. Trace:
I20260505 07:54:10.017035 2276 ts_tablet_manager.cc:1929] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:10.009456 2405 consensus_queue.cc:237] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.017387 2276 log.cc:1199] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/23fa401bb4634022bca3c6b23421e4d5
I20260505 07:54:10.018085 2276 ts_tablet_manager.cc:1950] T 23fa401bb4634022bca3c6b23421e4d5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:10.018035 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:10.015534 1353 rpcz_store.cc:276] 0505 07:54:08.092615 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.938320 (+1845705us) service_pool.cc:224] Handling call
0505 07:54:09.940423 (+ 2103us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:09.941327 (+ 904us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:09.986293 (+ 44966us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.986350 (+ 57us) write_op.cc:454] Released schema lock
0505 07:54:09.986742 (+ 392us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:09.987506 (+ 764us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:09.989413 (+ 1907us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.943771 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.943866 (+ 95us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.943871 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:09.943873 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:09.943947 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.943959 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.943964 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.944025 (+ 61us) tablet.cc:701] Row locks acquired
0505 07:54:09.944028 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:09.944055 (+ 27us) write_op.cc:270] Start()
0505 07:54:09.944081 (+ 26us) write_op.cc:276] Timestamp: P: 1777967649944012 usec, L: 0
0505 07:54:09.944084 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.944223 (+ 139us) log.cc:844] Serialized 433 byte log entry
0505 07:54:09.947801 (+ 3578us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.964628 (+ 16827us) write_op.cc:301] APPLY: starting
0505 07:54:09.964958 (+ 330us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.965326 (+ 368us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.965676 (+ 350us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.966984 (+ 1308us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.967455 (+ 471us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:09.967756 (+ 301us) write_op.cc:312] APPLY: finished
0505 07:54:09.968628 (+ 872us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.969249 (+ 621us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.970516 (+ 1267us) write_op.cc:454] Released schema lock
0505 07:54:09.970523 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":15697,"apply.run_cpu_time_us":806,"apply.run_wall_time_us":7610,"num_ops":2,"prepare.queue_time_us":79,"prepare.run_cpu_time_us":544,"prepare.run_wall_time_us":544,"raft.queue_time_us":432,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":275,"replication_time_us":3691}]]}
I20260505 07:54:10.024312 991 leader_election.cc:304] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.027202 2407 raft_consensus.cc:2804] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:10.027295 2407 raft_consensus.cc:493] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:10.027381 2407 raft_consensus.cc:3060] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.030488 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 69) took 2381 ms. Trace:
I20260505 07:54:10.030611 2276 rpcz_store.cc:276] 0505 07:54:07.648818 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.649154 (+ 336us) service_pool.cc:224] Handling call
0505 07:54:09.983321 (+2334167us) ts_tablet_manager.cc:1151] Deleting tablet 23fa401bb4634022bca3c6b23421e4d5
0505 07:54:09.983330 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.999807 (+ 16477us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.002830 (+ 3023us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.030472 (+ 27642us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2334006}
I20260505 07:54:10.031040 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 23fa401bb4634022bca3c6b23421e4d5 (table default.Table_0562bf620b5f46ecab060957ac1ca8bf [id=f47382441b814a62b5ff875ae632b915]) successfully deleted
I20260505 07:54:10.031634 1353 master_service.cc:954] ReplaceTablet: received request to replace tablet 26f3544e74d941d286a2a0317b941ab6 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:10.024683 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:10.032609 1181 tablet_bootstrap.cc:492] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:10.032735 1181 ts_tablet_manager.cc:1403] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.082s user 0.001s sys 0.002s
I20260505 07:54:10.033059 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:10.034121 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:10.034862 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:10.035951 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:10.036836 2276 tablet_replica.cc:333] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:10.036630 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:10.037106 2276 raft_consensus.cc:2243] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.037313 2276 raft_consensus.cc:2272] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.038362 2276 ts_tablet_manager.cc:1916] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.039179 1172 tablet_bootstrap.cc:492] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:10.039394 1172 ts_tablet_manager.cc:1403] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.047s user 0.007s sys 0.001s
I20260505 07:54:10.039251 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:10.039685 2505 raft_consensus.cc:493] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.039799 2505 raft_consensus.cc:515] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.040356 2505 leader_election.cc:290] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.040743 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f869abaa31e414ea0625e475db08e64" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:10.041435 854 leader_election.cc:343] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.041390 1172 raft_consensus.cc:359] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.041527 1172 raft_consensus.cc:385] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.041594 1172 raft_consensus.cc:740] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.041821 1172 consensus_queue.cc:260] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.042451 1172 ts_tablet_manager.cc:1434] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260505 07:54:10.043279 1172 tablet_bootstrap.cc:492] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:10.045517 1176 tablet_bootstrap.cc:492] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:10.045879 2505 raft_consensus.cc:493] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.045979 1176 ts_tablet_manager.cc:1403] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.054s user 0.005s sys 0.001s
I20260505 07:54:10.045996 2505 raft_consensus.cc:515] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.046561 2505 leader_election.cc:290] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.047348 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe5f6f0774545b296bf1b03a08e145d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.047528 1176 raft_consensus.cc:359] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:10.047883 854 leader_election.cc:343] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.047899 1176 raft_consensus.cc:385] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.048023 1176 raft_consensus.cc:740] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.048372 1176 consensus_queue.cc:260] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.049403 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:10.051671 1172 tablet_bootstrap.cc:654] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.051550 1176 ts_tablet_manager.cc:1434] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:10.053840 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:10.056376 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:10.059073 2276 ts_tablet_manager.cc:1929] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:10.061465 2268 ts_tablet_manager.cc:1929] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:10.064122 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f869abaa31e414ea0625e475db08e64" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.066422 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:10.068147 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]). Expected version 2 got 0
I20260505 07:54:10.070663 1176 tablet_bootstrap.cc:492] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:10.077837 2268 log.cc:1199] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/85d485ae2d1d4d76a6a5d988cec8291a
I20260505 07:54:10.079062 1181 raft_consensus.cc:359] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.079259 1181 raft_consensus.cc:385] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.079321 1181 raft_consensus.cc:740] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.079458 1176 tablet_bootstrap.cc:654] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.079507 1181 consensus_queue.cc:260] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.080041 1181 ts_tablet_manager.cc:1434] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.047s user 0.001s sys 0.000s
I20260505 07:54:10.080309 1181 tablet_bootstrap.cc:492] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:10.080083 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:10.081650 2407 raft_consensus.cc:515] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.082502 2268 ts_tablet_manager.cc:1950] T 85d485ae2d1d4d76a6a5d988cec8291a P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:10.077025 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe5f6f0774545b296bf1b03a08e145d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.083966 2407 leader_election.cc:290] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.084328 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 46) took 3001 ms. Trace:
I20260505 07:54:10.084474 2268 rpcz_store.cc:276] 0505 07:54:07.083072 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.114858 (+ 31786us) service_pool.cc:224] Handling call
0505 07:54:09.915941 (+2801083us) ts_tablet_manager.cc:1151] Deleting tablet 85d485ae2d1d4d76a6a5d988cec8291a
0505 07:54:09.915948 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:09.990994 (+ 75046us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.061399 (+ 70405us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.084315 (+ 22916us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2800962}
I20260505 07:54:10.084981 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 47) took 3001 ms. Trace:
I20260505 07:54:10.085104 2268 rpcz_store.cc:276] 0505 07:54:07.083206 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.114990 (+ 31784us) service_pool.cc:224] Handling call
0505 07:54:10.084879 (+2969889us) ts_tablet_manager.cc:1151] Deleting tablet e6a5f823db6143319d743d363a77ea47
0505 07:54:10.084888 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.084966 (+ 78us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2969802}
I20260505 07:54:10.085434 2268 tablet_replica.cc:333] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:10.085431 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f121fdd001ec4ea38ccd24a6afd9e51c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:10.085672 2268 raft_consensus.cc:2243] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.085819 2268 raft_consensus.cc:2272] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.085763 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
W20260505 07:54:10.086258 988 leader_election.cc:343] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.086337 2268 ts_tablet_manager.cc:1916] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.101207 1181 tablet_bootstrap.cc:654] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.104641 1176 tablet_bootstrap.cc:492] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:10.104910 1176 ts_tablet_manager.cc:1403] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.035s user 0.006s sys 0.000s
I20260505 07:54:10.106635 2535 tablet.cc:1724] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:10.106832 1176 raft_consensus.cc:359] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.107142 1176 raft_consensus.cc:385] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.107266 1176 raft_consensus.cc:740] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.107625 1176 consensus_queue.cc:260] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.110529 1176 ts_tablet_manager.cc:1434] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260505 07:54:10.113441 1176 tablet_bootstrap.cc:492] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:10.114789 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:10.104962 2447 ts_tablet_manager.cc:1929] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:10.117082 2447 log.cc:1199] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/aab3850799254dda9716fd5bf9404159
I20260505 07:54:10.117444 2447 ts_tablet_manager.cc:1950] T aab3850799254dda9716fd5bf9404159 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
W20260505 07:54:10.117789 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e6a5f823db6143319d743d363a77ea47 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e6a5f823db6143319d743d363a77ea47 already in progress: creating tablet
I20260505 07:54:10.118515 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet aab3850799254dda9716fd5bf9404159 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]) successfully deleted
I20260505 07:54:10.118757 2447 tablet_replica.cc:333] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
W20260505 07:54:10.118753 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 02e06d8815034d91986cef12964860ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02e06d8815034d91986cef12964860ea already in progress: creating tablet
W20260505 07:54:10.118988 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet abe5f6f0774545b296bf1b03a08e145d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abe5f6f0774545b296bf1b03a08e145d already in progress: creating tablet
W20260505 07:54:10.119172 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 472441bd29704be59aabb0f7a4964bb0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 472441bd29704be59aabb0f7a4964bb0 already in progress: creating tablet
W20260505 07:54:10.119352 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet df1b62a30b2842b897a04c793bb5e7ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df1b62a30b2842b897a04c793bb5e7ce already in progress: creating tablet
I20260505 07:54:10.121852 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:10.123999 2407 raft_consensus.cc:493] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.124104 2407 raft_consensus.cc:515] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.124668 2407 leader_election.cc:290] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.124488 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:10.125528 775 raft_consensus.cc:3060] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.126384 2447 raft_consensus.cc:2243] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.126535 2447 raft_consensus.cc:2272] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.127720 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feeafd614b3e4281a8e45b950e699b63" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:10.129133 988 leader_election.cc:343] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.129285 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:10.131562 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 42360f8ee4ff47c2af8d287f792207de (table default.Table_829cc346ac834637ab2e2198944a8550 [id=2c91868ddba84103b9a5c78201842a9e]). Expected version 1 got 0
I20260505 07:54:10.133249 2447 ts_tablet_manager.cc:1916] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.133714 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:10.137804 1176 tablet_bootstrap.cc:654] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.138511 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:10.142689 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:10.143991 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 21480c42b73e4bf0ab2453b085a867c0 (table default.Table_925028e721e24ba68f31fe1040d4c2fd [id=0faa6676bab04919984b580299143271]). Expected version 1 got 0
I20260505 07:54:10.146664 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:10.150060 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:10.151386 771 raft_consensus.cc:3060] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.155234 1181 tablet_bootstrap.cc:492] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:10.155424 1181 ts_tablet_manager.cc:1403] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.075s user 0.002s sys 0.008s
I20260505 07:54:10.156260 1181 raft_consensus.cc:359] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.156476 1181 raft_consensus.cc:385] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.156592 1181 raft_consensus.cc:740] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.156855 1181 consensus_queue.cc:260] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.157478 1181 ts_tablet_manager.cc:1434] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:10.166407 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:10.063102 2542 tablet.cc:1724] T 62bf3a6bc0ac42a99a4181f820e3ca87 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:10.168071 2276 log.cc:1199] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/9044763bed1a41739ecce4def4cca1fe
I20260505 07:54:10.171938 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]). Expected version 1 got 0
I20260505 07:54:10.113001 967 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:10.174149 1176 tablet_bootstrap.cc:492] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:10.174921 2276 ts_tablet_manager.cc:1950] T 9044763bed1a41739ecce4def4cca1fe P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:10.175271 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 70) took 2526 ms. Trace:
I20260505 07:54:10.175365 2276 rpcz_store.cc:276] 0505 07:54:07.648923 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.649331 (+ 408us) service_pool.cc:224] Handling call
0505 07:54:10.036789 (+2387458us) ts_tablet_manager.cc:1151] Deleting tablet 9044763bed1a41739ecce4def4cca1fe
0505 07:54:10.036799 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.054046 (+ 17247us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.059028 (+ 4982us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.175210 (+116182us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2387366}
I20260505 07:54:10.175817 2276 tablet_replica.cc:333] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:10.116218 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f121fdd001ec4ea38ccd24a6afd9e51c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:10.176023 2276 raft_consensus.cc:2243] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.176093 921 raft_consensus.cc:3060] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.176196 2276 raft_consensus.cc:2272] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.176707 2276 ts_tablet_manager.cc:1916] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.175498 1176 ts_tablet_manager.cc:1403] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.000s
I20260505 07:54:10.178903 921 raft_consensus.cc:2468] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:10.134918 2505 raft_consensus.cc:493] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.179191 2505 raft_consensus.cc:515] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.143576 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feeafd614b3e4281a8e45b950e699b63" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.179793 912 raft_consensus.cc:2468] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:10.067023 856 leader_election.cc:343] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.180008 856 leader_election.cc:304] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.180214 2444 raft_consensus.cc:2749] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:10.180305 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 113260us
I20260505 07:54:10.179908 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]). Expected version 2 got 0
W20260505 07:54:10.180728 856 leader_election.cc:343] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.181358 856 leader_election.cc:304] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.181592 2444 raft_consensus.cc:2749] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.181859 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 9044763bed1a41739ecce4def4cca1fe (table default.Table_1523b3ae7cd94c599e60a187bc677fc0 [id=bd575286f4e94df3b335b5e79d8f971c]) successfully deleted
I20260505 07:54:10.183329 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:10.185243 2505 leader_election.cc:290] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.186316 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:10.186115 1176 raft_consensus.cc:359] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.187691 1176 raft_consensus.cc:385] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.188247 1176 raft_consensus.cc:740] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.189829 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b9e8eb767a4eceb583ff9b47000d58" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.190012 1050 raft_consensus.cc:2468] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:10.190416 856 leader_election.cc:304] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:10.190726 2505 raft_consensus.cc:2804] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:10.190831 2505 raft_consensus.cc:493] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:10.190910 2505 raft_consensus.cc:3060] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.190446 1176 consensus_queue.cc:260] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.191758 991 leader_election.cc:304] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.192153 991 leader_election.cc:304] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.192514 2405 raft_consensus.cc:2804] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:10.192679 2302 raft_consensus.cc:2804] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:10.192751 2302 raft_consensus.cc:493] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:10.192837 2302 raft_consensus.cc:3060] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.192132 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:10.193426 2505 raft_consensus.cc:515] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.194334 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b9e8eb767a4eceb583ff9b47000d58" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:10.194479 781 raft_consensus.cc:3060] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.194470 2447 ts_tablet_manager.cc:1929] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:10.195191 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b9e8eb767a4eceb583ff9b47000d58" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:10.194566 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:10.195325 1050 raft_consensus.cc:3060] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.195693 2447 log.cc:1199] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/297537a92684425d8508f7b3b4c9d379
I20260505 07:54:10.197021 781 raft_consensus.cc:2468] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:10.196861 1176 ts_tablet_manager.cc:1434] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.018s user 0.003s sys 0.000s
I20260505 07:54:10.197934 1050 raft_consensus.cc:2468] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:10.197753 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_728dc87e78954a7fbbf495d74705e382 [id=2c06ff8eed31432e87f08e9b59049652]). Expected version 2 got 0
I20260505 07:54:10.198925 2302 raft_consensus.cc:515] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.199553 2302 leader_election.cc:290] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.199353 2447 ts_tablet_manager.cc:1950] T 297537a92684425d8508f7b3b4c9d379 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:10.199745 2405 raft_consensus.cc:697] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:10.200022 2405 consensus_queue.cc:237] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.198902 854 leader_election.cc:304] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:10.201462 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:10.201596 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feeafd614b3e4281a8e45b950e699b63" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:10.202220 912 raft_consensus.cc:3060] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.202288 2444 raft_consensus.cc:2804] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:10.202445 2444 raft_consensus.cc:697] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:10.194643 2505 leader_election.cc:290] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.202719 2444 consensus_queue.cc:237] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.203404 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
I20260505 07:54:10.205087 912 raft_consensus.cc:2468] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:10.205624 991 leader_election.cc:304] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:10.205978 1878 raft_consensus.cc:2804] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:10.206115 1878 raft_consensus.cc:697] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:10.206353 1878 consensus_queue.cc:237] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.208796 1172 tablet_bootstrap.cc:492] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:10.208994 1172 ts_tablet_manager.cc:1403] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.166s user 0.000s sys 0.006s
I20260505 07:54:10.209877 1172 raft_consensus.cc:359] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.210332 1172 raft_consensus.cc:385] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.210919 1172 raft_consensus.cc:740] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.211064 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0b9e8eb767a4eceb583ff9b47000d58" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.211292 775 raft_consensus.cc:2376] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1. Re-sending same reply.
I20260505 07:54:10.212262 1181 tablet_bootstrap.cc:492] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:10.212689 1172 consensus_queue.cc:260] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.213594 1353 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 131) took 1498 ms. Trace:
I20260505 07:54:10.213713 1353 rpcz_store.cc:276] 0505 07:54:08.715221 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.031456 (+1316235us) service_pool.cc:224] Handling call
0505 07:54:10.213549 (+182093us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:10.214295 1172 ts_tablet_manager.cc:1434] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:54:10.217885 2414 raft_consensus.cc:493] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:10.218035 1181 tablet_bootstrap.cc:654] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.218022 2414 raft_consensus.cc:515] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:10.218528 2414 leader_election.cc:290] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.219035 1172 tablet_bootstrap.cc:492] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:10.219534 717 leader_election.cc:343] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: ccdde3e5a13e436d9f2328831e155adb
I20260505 07:54:10.221264 1172 tablet_bootstrap.cc:654] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:10.231665 2257 log.cc:927] Time spent T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Append to log took a long time: real 0.064s user 0.001s sys 0.000s
I20260505 07:54:10.237526 2268 ts_tablet_manager.cc:1929] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:10.237659 2268 log.cc:1199] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/4662d4a3d88d4c2883a35048d6e11eb9
I20260505 07:54:10.238029 2268 ts_tablet_manager.cc:1950] T 4662d4a3d88d4c2883a35048d6e11eb9 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:10.238371 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 58) took 2765 ms. Trace:
I20260505 07:54:10.238462 2268 rpcz_store.cc:276] 0505 07:54:07.472764 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.472809 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:10.085390 (+2612581us) ts_tablet_manager.cc:1151] Deleting tablet 4662d4a3d88d4c2883a35048d6e11eb9
0505 07:54:10.085400 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.169248 (+ 83848us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.237468 (+ 68220us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.238356 (+ 888us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2612374}
I20260505 07:54:10.238881 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 4662d4a3d88d4c2883a35048d6e11eb9 (table default.Table_fcbd7c6e6fd44f3da96c6dded6eca3b9 [id=3aa673bdfdb2432bb2174d4a520d7a9e]) successfully deleted
I20260505 07:54:10.243665 2268 tablet_replica.cc:333] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:10.243871 2268 raft_consensus.cc:2243] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.244043 2268 raft_consensus.cc:2272] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.244241 1176 tablet_bootstrap.cc:492] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:10.244580 2268 ts_tablet_manager.cc:1916] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.247849 1176 tablet_bootstrap.cc:654] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.251160 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 297537a92684425d8508f7b3b4c9d379 (table default.Table_44ead87f92584e6e98e2c5f69056591a [id=e78409bbbd334198b96cc757bc1feee8]) successfully deleted
I20260505 07:54:10.251693 2447 tablet_replica.cc:333] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:10.252265 2447 raft_consensus.cc:2243] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.252661 2447 raft_consensus.cc:2272] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.253266 2276 ts_tablet_manager.cc:1929] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:10.254323 2447 ts_tablet_manager.cc:1916] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.274485 827 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:10.276049 2276 log.cc:1199] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/a7fe9c7199cc4039ae877dc3d4aab17e
I20260505 07:54:10.276605 2276 ts_tablet_manager.cc:1950] T a7fe9c7199cc4039ae877dc3d4aab17e P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:10.277004 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 65) took 2628 ms. Trace:
I20260505 07:54:10.277173 2276 rpcz_store.cc:276] 0505 07:54:07.648478 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.649628 (+ 1150us) service_pool.cc:224] Handling call
0505 07:54:10.175651 (+2526023us) ts_tablet_manager.cc:1151] Deleting tablet a7fe9c7199cc4039ae877dc3d4aab17e
0505 07:54:10.175736 (+ 85us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.250316 (+ 74580us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.253183 (+ 2867us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.276973 (+ 23790us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2525885}
I20260505 07:54:10.277675 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet a7fe9c7199cc4039ae877dc3d4aab17e (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]) successfully deleted
I20260505 07:54:10.277763 2276 tablet_replica.cc:333] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:10.278003 2276 raft_consensus.cc:2243] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:10.278378 2276 raft_consensus.cc:2272] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.282698 2307 consensus_queue.cc:1048] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:10.285929 1176 tablet_bootstrap.cc:492] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:10.286298 1176 ts_tablet_manager.cc:1403] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.042s user 0.002s sys 0.003s
I20260505 07:54:10.287741 1172 tablet_bootstrap.cc:492] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:10.287917 1172 ts_tablet_manager.cc:1403] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.069s user 0.004s sys 0.003s
I20260505 07:54:10.288671 1172 raft_consensus.cc:359] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.288869 1172 raft_consensus.cc:385] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.288949 1172 raft_consensus.cc:740] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.288717 1176 raft_consensus.cc:359] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.289213 1172 consensus_queue.cc:260] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.289252 1176 raft_consensus.cc:385] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.289438 1176 raft_consensus.cc:740] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.289825 1172 ts_tablet_manager.cc:1434] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:10.290115 1172 tablet_bootstrap.cc:492] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:10.289954 1176 consensus_queue.cc:260] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.291050 1176 ts_tablet_manager.cc:1434] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:10.291646 1176 tablet_bootstrap.cc:492] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:10.291801 1172 tablet_bootstrap.cc:654] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.294744 2268 ts_tablet_manager.cc:1929] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:10.296293 2276 ts_tablet_manager.cc:1916] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.296412 2089 raft_consensus.cc:493] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:10.296538 2089 raft_consensus.cc:515] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:10.297123 2089 leader_election.cc:290] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.324471 1176 tablet_bootstrap.cc:654] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.329075 2268 log.cc:1199] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/e278b86e7b5d444ebad8d0f6921d4c62
I20260505 07:54:10.337672 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:10.340365 2447 ts_tablet_manager.cc:1929] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:10.349671 2268 ts_tablet_manager.cc:1950] T e278b86e7b5d444ebad8d0f6921d4c62 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:10.349824 2444 raft_consensus.cc:493] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.349961 2444 raft_consensus.cc:515] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.350504 2444 leader_election.cc:290] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.350608 1181 tablet_bootstrap.cc:492] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:10.350732 1181 ts_tablet_manager.cc:1403] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.139s user 0.004s sys 0.003s
I20260505 07:54:10.350970 1357 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 5683d7366de84653ace56f0edcd3818c
I20260505 07:54:10.351280 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb7b3fded5d4def8db8d5c1e77ddcda" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.352003 2447 log.cc:1199] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/6ca2852cee84476dbe5e5072a43087a1
W20260505 07:54:10.352064 854 leader_election.cc:343] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.352154 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 5683d7366de84653ace56f0edcd3818c with delete_type TABLET_DATA_DELETED (replaced by tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.352212 1357 catalog_manager.cc:6741] ReplaceTablet: tablet 5683d7366de84653ace56f0edcd3818c deleted and replaced by tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6
I20260505 07:54:10.352553 2447 ts_tablet_manager.cc:1950] T 6ca2852cee84476dbe5e5072a43087a1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:10.352476 1357 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 128) took 2290 ms. Trace:
I20260505 07:54:10.352643 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 5683d7366de84653ace56f0edcd3818c with delete_type TABLET_DATA_DELETED (replaced by tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.353516 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 6ca2852cee84476dbe5e5072a43087a1 (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]) successfully deleted
I20260505 07:54:10.352685 1357 rpcz_store.cc:276] 0505 07:54:08.062317 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.879926 (+1817609us) service_pool.cc:224] Handling call
0505 07:54:09.902161 (+ 22235us) spinlock_profiling.cc:243] Waited 21.9 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639e385b 00007fd65a477107 00007fd65a476e35 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:10.352396 (+450235us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.228095 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.228221 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.228227 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:10.228230 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:10.228347 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.228361 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.228365 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:10.228442 (+ 77us) tablet.cc:701] Row locks acquired
0505 07:54:10.228444 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:10.228478 (+ 34us) write_op.cc:270] Start()
0505 07:54:10.228515 (+ 37us) write_op.cc:276] Timestamp: P: 1777967650228456 usec, L: 0
0505 07:54:10.228521 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.228719 (+ 198us) log.cc:844] Serialized 656 byte log entry
0505 07:54:10.231810 (+ 3091us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.231928 (+ 118us) write_op.cc:301] APPLY: starting
0505 07:54:10.231943 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.231986 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.232016 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.232106 (+ 90us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.232157 (+ 51us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:10.232165 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:10.232232 (+ 67us) log.cc:844] Serialized 29 byte log entry
0505 07:54:10.232291 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.232310 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:10.232316 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:10.232382 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.232386 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":50359168,"child_traces":[["op",{"apply.queue_time_us":65,"apply.run_cpu_time_us":446,"apply.run_wall_time_us":446,"num_ops":2,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":719,"prepare.run_wall_time_us":719,"raft.queue_time_us":453,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":265,"replication_time_us":3263}]]}
I20260505 07:54:10.354375 1176 tablet_bootstrap.cc:492] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:10.354498 1176 ts_tablet_manager.cc:1403] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.063s user 0.004s sys 0.000s
I20260505 07:54:10.355183 1176 raft_consensus.cc:359] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.355352 1176 raft_consensus.cc:385] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.355404 1176 raft_consensus.cc:740] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.354786 1357 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_5fcf5ea766444ab5b4515e76b4db6cfd" } new_table_name: "default.Table_e7ee181e7c8e4cd9a155996757c94903" modify_external_catalogs: true
I20260505 07:54:10.355609 1176 consensus_queue.cc:260] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.355690 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 5683d7366de84653ace56f0edcd3818c with delete_type TABLET_DATA_DELETED (replaced by tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:10.356297 717 leader_election.cc:343] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 6da10815e0dc41a1a07b6786882fffd3
I20260505 07:54:10.356685 1176 ts_tablet_manager.cc:1434] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:10.357497 1176 tablet_bootstrap.cc:492] T e79a1da502604d05a62e27f575b8b8f4 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:10.359380 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb7b3fded5d4def8db8d5c1e77ddcda" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.359552 1050 raft_consensus.cc:2468] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:10.359958 856 leader_election.cc:304] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.362176 2444 raft_consensus.cc:2804] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:10.362259 2444 raft_consensus.cc:493] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:10.362324 2444 raft_consensus.cc:3060] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.363930 1181 raft_consensus.cc:359] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.364064 1181 raft_consensus.cc:385] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.364117 1181 raft_consensus.cc:740] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.364282 1181 consensus_queue.cc:260] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.364748 1181 ts_tablet_manager.cc:1434] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.014s user 0.000s sys 0.000s
I20260505 07:54:10.364971 1181 tablet_bootstrap.cc:492] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:10.364926 2444 raft_consensus.cc:515] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.365492 2444 leader_election.cc:290] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.366581 1181 tablet_bootstrap.cc:654] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.367139 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb7b3fded5d4def8db8d5c1e77ddcda" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:10.367677 854 leader_election.cc:343] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.368964 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cb7b3fded5d4def8db8d5c1e77ddcda" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:10.369123 1050 raft_consensus.cc:3060] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.371946 1050 raft_consensus.cc:2468] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:10.372390 856 leader_election.cc:304] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.372866 2444 raft_consensus.cc:2804] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:10.373122 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 61) took 2648 ms. Trace:
I20260505 07:54:10.373250 2268 rpcz_store.cc:276] 0505 07:54:07.724678 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.726800 (+ 2122us) service_pool.cc:224] Handling call
0505 07:54:10.243618 (+2516818us) ts_tablet_manager.cc:1151] Deleting tablet e278b86e7b5d444ebad8d0f6921d4c62
0505 07:54:10.243629 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.248154 (+ 4525us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.294685 (+ 46531us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.373105 (+ 78420us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2516606}
I20260505 07:54:10.373355 2444 raft_consensus.cc:697] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:10.373589 2444 consensus_queue.cc:237] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.373744 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet e278b86e7b5d444ebad8d0f6921d4c62 (table default.Table_633da5b39da34f5c88163c9936fd0212 [id=d53c653124364ea5acc357977c9db864]) successfully deleted
I20260505 07:54:10.374190 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 59) took 2649 ms. Trace:
I20260505 07:54:10.374301 2268 rpcz_store.cc:276] 0505 07:54:07.724462 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.731779 (+ 7317us) service_pool.cc:224] Handling call
0505 07:54:10.374065 (+2642286us) ts_tablet_manager.cc:1151] Deleting tablet 54e4f7f895d943639653c8c32724baf2
0505 07:54:10.374074 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.374176 (+ 102us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2642034}
W20260505 07:54:10.374733 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 54e4f7f895d943639653c8c32724baf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54e4f7f895d943639653c8c32724baf2 already in progress: creating tablet
I20260505 07:54:10.375988 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 64) took 2264 ms. Trace:
I20260505 07:54:10.376125 2268 rpcz_store.cc:276] 0505 07:54:08.111989 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:08.112048 (+ 59us) service_pool.cc:224] Handling call
0505 07:54:10.375850 (+2263802us) ts_tablet_manager.cc:1151] Deleting tablet 25072055f5534078adf5b5025614f951
0505 07:54:10.375860 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.375974 (+ 114us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2263607}
W20260505 07:54:10.376600 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 25072055f5534078adf5b5025614f951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25072055f5534078adf5b5025614f951 already in progress: creating tablet
I20260505 07:54:10.376897 2268 tablet_replica.cc:333] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:10.377584 2268 raft_consensus.cc:2243] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:10.377915 2268 raft_consensus.cc:2272] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.378473 2268 ts_tablet_manager.cc:1916] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.380718 1176 tablet_bootstrap.cc:654] T e79a1da502604d05a62e27f575b8b8f4 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:10.381866 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet ae829daad64545e687a9528e5c38b9ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae829daad64545e687a9528e5c38b9ed already in progress: creating tablet
W20260505 07:54:10.382534 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet f200abef13ac42698a9b96fc8af82ff0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f200abef13ac42698a9b96fc8af82ff0 already in progress: creating tablet
I20260505 07:54:10.387902 2505 raft_consensus.cc:493] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:10.388031 2505 raft_consensus.cc:515] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:10.388589 2505 leader_election.cc:290] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:10.389074 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.389227 771 raft_consensus.cc:2468] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:10.389638 854 leader_election.cc:304] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:10.390076 2505 raft_consensus.cc:2804] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:10.390146 2505 raft_consensus.cc:493] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:10.390218 2505 raft_consensus.cc:3060] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Advancing to term 2
W20260505 07:54:10.390918 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 54e4f7f895d943639653c8c32724baf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 54e4f7f895d943639653c8c32724baf2 already in progress: creating tablet
I20260505 07:54:10.392556 2505 raft_consensus.cc:515] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:10.393067 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 114) took 1003 ms. Trace:
I20260505 07:54:10.393492 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:10.393647 771 raft_consensus.cc:3060] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:10.393850 2447 rpcz_store.cc:276] 0505 07:54:09.389620 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.392369 (+ 2749us) service_pool.cc:224] Handling call
0505 07:54:10.392977 (+1000608us) ts_tablet_manager.cc:1151] Deleting tablet accb2dedc30e494ebd22c33594cee661
0505 07:54:10.392986 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.393052 (+ 66us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1000389}
W20260505 07:54:10.394238 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet accb2dedc30e494ebd22c33594cee661 because the tablet was not found. No further retry: Not found: Tablet not found: accb2dedc30e494ebd22c33594cee661
I20260505 07:54:10.394513 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 122) took 1001 ms. Trace:
I20260505 07:54:10.394572 2447 rpcz_store.cc:276] 0505 07:54:09.392818 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.392864 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:10.394404 (+1001540us) ts_tablet_manager.cc:1151] Deleting tablet ba616702dc6040bb805bb3ffc8a8a871
0505 07:54:10.394412 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.394498 (+ 86us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1001375}
W20260505 07:54:10.394917 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet ba616702dc6040bb805bb3ffc8a8a871 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba616702dc6040bb805bb3ffc8a8a871 already in progress: creating tablet
I20260505 07:54:10.396050 771 raft_consensus.cc:2468] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 2.
I20260505 07:54:10.396217 2447 tablet_replica.cc:333] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:10.396389 2447 raft_consensus.cc:2243] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:10.396389 854 leader_election.cc:304] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:10.396477 757 tablet_service.cc:1511] Processing CreateTablet for tablet 2f761934fef541a7b53eeda1f1cb464c (DEFAULT_TABLE table=default.Table_4d8d7c4d5cc546a184cd68a8c7979589 [id=625dd23752b2427581455de96dcb793b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.396646 2392 raft_consensus.cc:2804] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:10.396639 2447 raft_consensus.cc:2272] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.396848 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f761934fef541a7b53eeda1f1cb464c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.397147 2447 ts_tablet_manager.cc:1916] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:10.398042 856 leader_election.cc:343] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 6da10815e0dc41a1a07b6786882fffd3
I20260505 07:54:10.398758 2505 leader_election.cc:290] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:10.400126 856 leader_election.cc:343] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 6da10815e0dc41a1a07b6786882fffd3
I20260505 07:54:10.401163 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 2f761934fef541a7b53eeda1f1cb464c (DEFAULT_TABLE table=default.Table_4d8d7c4d5cc546a184cd68a8c7979589 [id=625dd23752b2427581455de96dcb793b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.401290 2414 raft_consensus.cc:493] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.401381 2414 raft_consensus.cc:515] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.401499 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f761934fef541a7b53eeda1f1cb464c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.401880 2414 leader_election.cc:290] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.402401 751 tablet_service.cc:1511] Processing CreateTablet for tablet d263e55f93894e59b18e78d2b36cba39 (DEFAULT_TABLE table=default.Table_3ffe224aa51546ed94df1af5344e3d0a [id=ce15d9ac1c504dbe9febe55f0abf0a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.402673 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d263e55f93894e59b18e78d2b36cba39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.403707 2302 raft_consensus.cc:493] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.403818 2302 raft_consensus.cc:515] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.404292 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4e5af08ddc4c77b5640b8295b236b1" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.404364 2302 leader_election.cc:290] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.404872 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79a1da502604d05a62e27f575b8b8f4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.405014 912 raft_consensus.cc:2468] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:10.405336 991 leader_election.cc:304] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:10.405598 2302 raft_consensus.cc:2804] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:10.405659 2302 raft_consensus.cc:493] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:10.405710 2302 raft_consensus.cc:3060] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.408107 2302 raft_consensus.cc:515] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.408684 2302 leader_election.cc:290] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.409273 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79a1da502604d05a62e27f575b8b8f4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:10.409464 921 raft_consensus.cc:3060] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.410353 899 tablet_service.cc:1511] Processing CreateTablet for tablet 2f761934fef541a7b53eeda1f1cb464c (DEFAULT_TABLE table=default.Table_4d8d7c4d5cc546a184cd68a8c7979589 [id=625dd23752b2427581455de96dcb793b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.410668 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f761934fef541a7b53eeda1f1cb464c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.412671 921 raft_consensus.cc:2468] T e79a1da502604d05a62e27f575b8b8f4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:10.415696 2558 raft_consensus.cc:493] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.415805 2558 raft_consensus.cc:515] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.415936 991 leader_election.cc:304] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:10.416309 1878 raft_consensus.cc:2804] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:10.416414 1878 raft_consensus.cc:697] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:10.416627 1878 consensus_queue.cc:237] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.416726 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca12a17fb014374bf7ed764a91e6985" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.417897 2558 leader_election.cc:290] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.418511 1034 tablet_service.cc:1511] Processing CreateTablet for tablet d263e55f93894e59b18e78d2b36cba39 (DEFAULT_TABLE table=default.Table_3ffe224aa51546ed94df1af5344e3d0a [id=ce15d9ac1c504dbe9febe55f0abf0a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.419056 751 tablet_service.cc:1511] Processing CreateTablet for tablet 9d64aafd59db4aa3a285a02c2cd7540b (DEFAULT_TABLE table=default.Table_98bcfc12727141509f5ccf7049009989 [id=3a2d6879f72f47bcaaecc3fd77d8cbd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.419436 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d64aafd59db4aa3a285a02c2cd7540b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.420663 2558 raft_consensus.cc:493] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.420771 2558 raft_consensus.cc:515] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.421262 2558 leader_election.cc:290] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.421967 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29ba55a846548679cb3b86305d0d6ed" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.422887 887 tablet_service.cc:1511] Processing CreateTablet for tablet d263e55f93894e59b18e78d2b36cba39 (DEFAULT_TABLE table=default.Table_3ffe224aa51546ed94df1af5344e3d0a [id=ce15d9ac1c504dbe9febe55f0abf0a69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.423185 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d263e55f93894e59b18e78d2b36cba39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.424788 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d263e55f93894e59b18e78d2b36cba39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.430502 1021 tablet_service.cc:1511] Processing CreateTablet for tablet 9d64aafd59db4aa3a285a02c2cd7540b (DEFAULT_TABLE table=default.Table_98bcfc12727141509f5ccf7049009989 [id=3a2d6879f72f47bcaaecc3fd77d8cbd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.430907 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d64aafd59db4aa3a285a02c2cd7540b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.430886 892 tablet_service.cc:1511] Processing CreateTablet for tablet 9d64aafd59db4aa3a285a02c2cd7540b (DEFAULT_TABLE table=default.Table_98bcfc12727141509f5ccf7049009989 [id=3a2d6879f72f47bcaaecc3fd77d8cbd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.431020 751 tablet_service.cc:1511] Processing CreateTablet for tablet 2898675aa97948d7a24b8e6d3f5258d0 (DEFAULT_TABLE table=default.Table_3f6be0e718b349a089b5aa8436614766 [id=5da56e2b8ebd48349a4caf6b59ab43e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.431212 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d64aafd59db4aa3a285a02c2cd7540b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.431313 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2898675aa97948d7a24b8e6d3f5258d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.436365 894 tablet_service.cc:1511] Processing CreateTablet for tablet 2898675aa97948d7a24b8e6d3f5258d0 (DEFAULT_TABLE table=default.Table_3f6be0e718b349a089b5aa8436614766 [id=5da56e2b8ebd48349a4caf6b59ab43e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.436679 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2898675aa97948d7a24b8e6d3f5258d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.439705 758 tablet_service.cc:1511] Processing CreateTablet for tablet 1bd5951bfc3346c2a6aad8f33357394a (DEFAULT_TABLE table=default.Table_0f05cd371317454b9be3fe9d7c31a9c4 [id=d85dae5f1f424012929e0d90ef8b2d22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.440001 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bd5951bfc3346c2a6aad8f33357394a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.447139 1021 tablet_service.cc:1511] Processing CreateTablet for tablet 1bd5951bfc3346c2a6aad8f33357394a (DEFAULT_TABLE table=default.Table_0f05cd371317454b9be3fe9d7c31a9c4 [id=d85dae5f1f424012929e0d90ef8b2d22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.447445 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bd5951bfc3346c2a6aad8f33357394a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.449875 894 tablet_service.cc:1511] Processing CreateTablet for tablet d0a9be9f3e084cb1ae57258c8c8e43eb (DEFAULT_TABLE table=default.Table_72dc49a729784b0cb6f4b62ee11fa2d0 [id=354b93ef3de344168cf2c6e79c16ea6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.450074 758 tablet_service.cc:1511] Processing CreateTablet for tablet b409a6f61ce2445ea5b94ccd8f61ebf8 (DEFAULT_TABLE table=default.Table_7ff5390dc98f47729200b3f768305c40 [id=6c9eca687b4c4708a760bc12f1b67fb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.450320 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a9be9f3e084cb1ae57258c8c8e43eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.450366 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b409a6f61ce2445ea5b94ccd8f61ebf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.450888 757 tablet_service.cc:1511] Processing CreateTablet for tablet d0a9be9f3e084cb1ae57258c8c8e43eb (DEFAULT_TABLE table=default.Table_72dc49a729784b0cb6f4b62ee11fa2d0 [id=354b93ef3de344168cf2c6e79c16ea6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.453637 899 tablet_service.cc:1511] Processing CreateTablet for tablet 1bd5951bfc3346c2a6aad8f33357394a (DEFAULT_TABLE table=default.Table_0f05cd371317454b9be3fe9d7c31a9c4 [id=d85dae5f1f424012929e0d90ef8b2d22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.453925 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bd5951bfc3346c2a6aad8f33357394a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.453922 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a9be9f3e084cb1ae57258c8c8e43eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.459657 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 2898675aa97948d7a24b8e6d3f5258d0 (DEFAULT_TABLE table=default.Table_3f6be0e718b349a089b5aa8436614766 [id=5da56e2b8ebd48349a4caf6b59ab43e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.460078 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2898675aa97948d7a24b8e6d3f5258d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.461967 1025 tablet_service.cc:1511] Processing CreateTablet for tablet b409a6f61ce2445ea5b94ccd8f61ebf8 (DEFAULT_TABLE table=default.Table_7ff5390dc98f47729200b3f768305c40 [id=6c9eca687b4c4708a760bc12f1b67fb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.462297 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b409a6f61ce2445ea5b94ccd8f61ebf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.462564 758 tablet_service.cc:1511] Processing CreateTablet for tablet 317144923d724f6fa96ad49809d25b46 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.462859 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317144923d724f6fa96ad49809d25b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.463279 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:10.464079 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:10.464725 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:10.465227 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:10.465734 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:10.466142 2447 ts_tablet_manager.cc:1929] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:10.466240 2447 log.cc:1199] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/872362aecf6a44b59e586b1fa7637c8b
I20260505 07:54:10.466538 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:10.466575 2447 ts_tablet_manager.cc:1950] T 872362aecf6a44b59e586b1fa7637c8b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:10.467141 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:10.467850 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:10.468727 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:10.469205 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:10.469235 1034 tablet_service.cc:1511] Processing CreateTablet for tablet d0a9be9f3e084cb1ae57258c8c8e43eb (DEFAULT_TABLE table=default.Table_72dc49a729784b0cb6f4b62ee11fa2d0 [id=354b93ef3de344168cf2c6e79c16ea6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.469616 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a9be9f3e084cb1ae57258c8c8e43eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.469910 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:10.470702 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:10.471804 894 tablet_service.cc:1511] Processing CreateTablet for tablet 317144923d724f6fa96ad49809d25b46 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.472287 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317144923d724f6fa96ad49809d25b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.475675 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 317144923d724f6fa96ad49809d25b46 (DEFAULT_TABLE table=default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.475960 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317144923d724f6fa96ad49809d25b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.479066 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:10.479967 1350 catalog_manager.cc:5671] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:10.481093 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
I20260505 07:54:10.482836 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 123) took 1089 ms. Trace:
I20260505 07:54:10.482939 2447 rpcz_store.cc:276] 0505 07:54:09.393215 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.393262 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:10.396176 (+1002914us) ts_tablet_manager.cc:1151] Deleting tablet 872362aecf6a44b59e586b1fa7637c8b
0505 07:54:10.396186 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.433135 (+ 36949us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.466101 (+ 32966us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.482798 (+ 16697us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1002769}
I20260505 07:54:10.483705 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 872362aecf6a44b59e586b1fa7637c8b (table default.Table_4d44519a55e0425d8016d1c0c28ed7f7 [id=4a43764f58e442ad892621ae1f29b214]) successfully deleted
I20260505 07:54:10.484076 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 124) took 1090 ms. Trace:
I20260505 07:54:10.484162 2447 rpcz_store.cc:276] 0505 07:54:09.393581 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.393628 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:10.483922 (+1090294us) ts_tablet_manager.cc:1151] Deleting tablet e7887c3462f74740b314e3bec36e280d
0505 07:54:10.483936 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.484056 (+ 120us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1089741}
W20260505 07:54:10.484609 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e7887c3462f74740b314e3bec36e280d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7887c3462f74740b314e3bec36e280d already in progress: creating tablet
I20260505 07:54:10.484913 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 125) took 1090 ms. Trace:
I20260505 07:54:10.484998 2447 rpcz_store.cc:276] 0505 07:54:09.393989 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.394036 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:10.484456 (+1090420us) ts_tablet_manager.cc:1151] Deleting tablet 727cacd242844455915bd9963d2db19f
0505 07:54:10.484467 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.484895 (+ 428us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1090237}
W20260505 07:54:10.485426 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 727cacd242844455915bd9963d2db19f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 727cacd242844455915bd9963d2db19f already in progress: creating tablet
I20260505 07:54:10.486007 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 126) took 1091 ms. Trace:
I20260505 07:54:10.486066 2447 rpcz_store.cc:276] 0505 07:54:09.394383 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.394429 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:10.485247 (+1090818us) ts_tablet_manager.cc:1151] Deleting tablet 6546745e894545ff9c5f91bcc6e1a41e
0505 07:54:10.485258 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.485993 (+ 735us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1090671}
W20260505 07:54:10.486367 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 6546745e894545ff9c5f91bcc6e1a41e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6546745e894545ff9c5f91bcc6e1a41e already in progress: creating tablet
I20260505 07:54:10.487197 2447 tablet_replica.cc:333] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:10.487399 2447 raft_consensus.cc:2243] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.487536 2447 raft_consensus.cc:2272] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.481712 1350 catalog_manager.cc:5671] T 934cea6666b1420eb12fe4510d2466fe P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:10.489548 2302 raft_consensus.cc:493] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.489705 2302 raft_consensus.cc:515] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.490419 2447 ts_tablet_manager.cc:1916] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.490598 2302 leader_election.cc:290] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.490980 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d442119970a345f28de029eb2139f5c1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.491147 921 raft_consensus.cc:2468] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:10.491230 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:10.491497 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_728dc87e78954a7fbbf495d74705e382 [id=2c06ff8eed31432e87f08e9b59049652]). Expected version 2 got 0
I20260505 07:54:10.491921 991 leader_election.cc:304] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:10.492147 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:10.492864 2302 raft_consensus.cc:2804] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:10.493196 2302 raft_consensus.cc:493] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:10.493577 2302 raft_consensus.cc:3060] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.494019 1350 catalog_manager.cc:5671] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:10.494477 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:10.495361 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:10.498520 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:10.499330 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:10.499842 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]). Expected version 2 got 0
I20260505 07:54:10.501389 1350 catalog_manager.cc:5671] T 036d306d592f41cb96814aa60e30551e P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:10.503419 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:10.504190 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:10.505180 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:10.505705 1172 tablet_bootstrap.cc:492] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:10.505846 1172 ts_tablet_manager.cc:1403] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.216s user 0.006s sys 0.001s
I20260505 07:54:10.505695 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:10.506467 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:10.506593 1172 raft_consensus.cc:359] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.506773 1172 raft_consensus.cc:385] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.506851 1172 raft_consensus.cc:740] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.507036 1172 consensus_queue.cc:260] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.507558 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:10.507655 1172 ts_tablet_manager.cc:1434] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:10.507918 1172 tablet_bootstrap.cc:492] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:10.509092 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 08c6ed755fc64080a2310fe9a4e0a68b (table default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b]). Expected version 3 got 0
I20260505 07:54:10.509459 1172 tablet_bootstrap.cc:654] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.509783 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feeafd614b3e4281a8e45b950e699b63" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:10.510197 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:10.510864 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:10.511547 771 raft_consensus.cc:3060] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.512079 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:10.512468 2447 ts_tablet_manager.cc:1929] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:10.512563 2447 log.cc:1199] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/5ad8a79245ad47738d079752e51aed2b
I20260505 07:54:10.511446 2302 raft_consensus.cc:515] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:10.512926 444 leader_election.cc:336] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401): Timed out: RequestConsensusVote RPC to 127.0.105.60:35401 timed out after 0.608s (SENT)
I20260505 07:54:10.513047 444 leader_election.cc:304] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 379cdfdecc3341ea9d7d9dc09274338d; no voters: 23c57a2c927c4c54bccecfeb3a775311, 28d3eda04f9c47ae976d19c218b6dcc5
I20260505 07:54:10.513552 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:10.514081 1350 catalog_manager.cc:5671] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:10.515045 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 21480c42b73e4bf0ab2453b085a867c0 (table default.Table_925028e721e24ba68f31fe1040d4c2fd [id=0faa6676bab04919984b580299143271]). Expected version 1 got 0
I20260505 07:54:10.516712 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:10.517495 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:10.518575 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]). Expected version 2 got 0
I20260505 07:54:10.519786 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:10.520378 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:10.520546 775 raft_consensus.cc:3060] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.521083 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:10.521848 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:10.522513 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:10.523061 1350 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]). Expected version 1 got 0
I20260505 07:54:10.524114 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79a1da502604d05a62e27f575b8b8f4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.524977 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e79a1da502604d05a62e27f575b8b8f4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:10.525807 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d442119970a345f28de029eb2139f5c1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.526296 781 raft_consensus.cc:3060] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.541159 2405 raft_consensus.cc:493] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.541301 2405 raft_consensus.cc:515] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.541864 2405 leader_election.cc:290] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.542752 2567 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260505 07:54:10.543738 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1602ccbae24544c4a6cf14ea073880b4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.543738 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d442119970a345f28de029eb2139f5c1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:10.543915 921 raft_consensus.cc:2468] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:10.543965 911 raft_consensus.cc:3060] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.546705 911 raft_consensus.cc:2468] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:10.547887 991 leader_election.cc:304] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:10.548724 2307 raft_consensus.cc:2804] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:10.548801 2307 raft_consensus.cc:493] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:10.548867 2307 raft_consensus.cc:3060] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.542721 2302 leader_election.cc:290] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.551435 2307 raft_consensus.cc:515] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.552784 991 leader_election.cc:304] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:10.553653 2438 raft_consensus.cc:2804] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:10.553767 2438 raft_consensus.cc:697] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:10.554028 2438 consensus_queue.cc:237] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.554900 2407 consensus_queue.cc:1048] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:10.557390 2392 raft_consensus.cc:697] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:10.557679 2392 consensus_queue.cc:237] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:10.559636 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e4e5af08ddc4c77b5640b8295b236b1" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.560652 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ca12a17fb014374bf7ed764a91e6985" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.560824 907 raft_consensus.cc:2393] T 2ca12a17fb014374bf7ed764a91e6985 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:10.561959 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29ba55a846548679cb3b86305d0d6ed" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.562630 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6da10815e0dc41a1a07b6786882fffd3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
W20260505 07:54:10.563311 988 leader_election.cc:343] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:10.563655 988 consensus_peers.cc:597] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ca2852cee84476dbe5e5072a43087a1. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:10.564280 2402 consensus_queue.cc:1048] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:10.564873 1181 tablet_bootstrap.cc:492] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:10.564989 1181 ts_tablet_manager.cc:1403] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.200s user 0.005s sys 0.005s
I20260505 07:54:10.565743 1181 raft_consensus.cc:359] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.565953 1181 raft_consensus.cc:385] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.566054 1181 raft_consensus.cc:740] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.566315 1181 consensus_queue.cc:260] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.566941 1181 ts_tablet_manager.cc:1434] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:10.567237 1181 tablet_bootstrap.cc:492] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:10.568867 1181 tablet_bootstrap.cc:654] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.569204 899 tablet_service.cc:1511] Processing CreateTablet for tablet b409a6f61ce2445ea5b94ccd8f61ebf8 (DEFAULT_TABLE table=default.Table_7ff5390dc98f47729200b3f768305c40 [id=6c9eca687b4c4708a760bc12f1b67fb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.569670 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b409a6f61ce2445ea5b94ccd8f61ebf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.582242 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1602ccbae24544c4a6cf14ea073880b4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:10.582428 911 raft_consensus.cc:3060] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.585180 911 raft_consensus.cc:2468] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:10.585734 757 tablet_service.cc:1511] Processing CreateTablet for tablet 5e08a8fd95c44b0ba52948e40b658bc9 (DEFAULT_TABLE table=default.Table_2eebf692e6ca4f6f8f7a1c82bbf85823 [id=a92d968444b84dc2b9ccc8660f3e5355]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.586144 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e08a8fd95c44b0ba52948e40b658bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.593410 894 tablet_service.cc:1511] Processing CreateTablet for tablet 5e08a8fd95c44b0ba52948e40b658bc9 (DEFAULT_TABLE table=default.Table_2eebf692e6ca4f6f8f7a1c82bbf85823 [id=a92d968444b84dc2b9ccc8660f3e5355]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.593722 751 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd95227fcf4470b880514caf1ac045a (DEFAULT_TABLE table=default.Table_c67b8faa7954415aa740a42b0915aa34 [id=5b776dd2a8c74b178b42ecfd9f0d4e82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.593840 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e08a8fd95c44b0ba52948e40b658bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.594112 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd95227fcf4470b880514caf1ac045a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.603075 894 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd95227fcf4470b880514caf1ac045a (DEFAULT_TABLE table=default.Table_c67b8faa7954415aa740a42b0915aa34 [id=5b776dd2a8c74b178b42ecfd9f0d4e82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.603250 751 tablet_service.cc:1511] Processing CreateTablet for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (DEFAULT_TABLE table=default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.603511 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd95227fcf4470b880514caf1ac045a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.603598 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.610338 887 tablet_service.cc:1511] Processing CreateTablet for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (DEFAULT_TABLE table=default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.610736 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:10.619390 718 leader_election.cc:343] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: ccdde3e5a13e436d9f2328831e155adb
I20260505 07:54:10.619599 718 leader_election.cc:304] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:10.619782 1176 tablet_bootstrap.cc:492] T e79a1da502604d05a62e27f575b8b8f4 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:10.619905 1176 ts_tablet_manager.cc:1403] T e79a1da502604d05a62e27f575b8b8f4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.262s user 0.015s sys 0.009s
I20260505 07:54:10.619930 2280 raft_consensus.cc:2749] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:10.620525 718 leader_election.cc:304] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:10.620622 1176 raft_consensus.cc:359] T e79a1da502604d05a62e27f575b8b8f4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.620774 2280 raft_consensus.cc:2749] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.620775 1176 raft_consensus.cc:385] T e79a1da502604d05a62e27f575b8b8f4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.620923 1176 raft_consensus.cc:740] T e79a1da502604d05a62e27f575b8b8f4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.620924 718 leader_election.cc:304] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:10.621176 2280 raft_consensus.cc:2749] T 2ca12a17fb014374bf7ed764a91e6985 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.621140 1176 consensus_queue.cc:260] T e79a1da502604d05a62e27f575b8b8f4 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.621348 718 leader_election.cc:304] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:10.621536 2089 raft_consensus.cc:2749] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.621608 1176 ts_tablet_manager.cc:1434] T e79a1da502604d05a62e27f575b8b8f4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:10.621747 718 leader_election.cc:304] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:10.623159 2447 ts_tablet_manager.cc:1950] T 5ad8a79245ad47738d079752e51aed2b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:10.623243 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:10.623512 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:10.623814 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:10.624073 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:10.624445 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:10.624656 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:10.624899 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:10.625195 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:10.625500 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:10.625836 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:10.626015 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:10.626180 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:10.626348 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:10.626516 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:10.627104 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:10.627311 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:10.627761 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
W20260505 07:54:10.571774 988 leader_election.cc:343] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.581729 2307 leader_election.cc:290] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.584226 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
W20260505 07:54:10.630203 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 58429.5us
I20260505 07:54:10.630528 1176 tablet_bootstrap.cc:492] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:54:10.630487 988 leader_election.cc:343] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:10.630864 988 leader_election.cc:343] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.632222 1176 tablet_bootstrap.cc:654] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.634125 1878 consensus_queue.cc:1048] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:10.585872 991 leader_election.cc:304] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:10.635365 2405 raft_consensus.cc:2804] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:10.635509 2405 raft_consensus.cc:697] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
W20260505 07:54:10.635749 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 49910.2us
I20260505 07:54:10.636473 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
I20260505 07:54:10.636620 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 5e08a8fd95c44b0ba52948e40b658bc9 (DEFAULT_TABLE table=default.Table_2eebf692e6ca4f6f8f7a1c82bbf85823 [id=a92d968444b84dc2b9ccc8660f3e5355]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.636684 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:10.636772 2405 consensus_queue.cc:237] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.636889 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 62bf3a6bc0ac42a99a4181f820e3ca87 (table default.Table_728dc87e78954a7fbbf495d74705e382 [id=2c06ff8eed31432e87f08e9b59049652]). Expected version 2 got 0
I20260505 07:54:10.637133 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e08a8fd95c44b0ba52948e40b658bc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.637421 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:10.637797 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:10.638124 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:10.638571 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:10.638638 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 8fd95227fcf4470b880514caf1ac045a (DEFAULT_TABLE table=default.Table_c67b8faa7954415aa740a42b0915aa34 [id=5b776dd2a8c74b178b42ecfd9f0d4e82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.638798 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:10.639050 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:10.639115 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fd95227fcf4470b880514caf1ac045a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.642864 1021 tablet_service.cc:1511] Processing CreateTablet for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (DEFAULT_TABLE table=default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:10.643247 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:10.643462 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:10.643438 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:10.643745 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:10.643944 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_5fcf5ea766444ab5b4515e76b4db6cfd [id=09bd9815349347ab9b75c4242c354c57]). Expected version 2 got 0
I20260505 07:54:10.644157 1172 tablet_bootstrap.cc:492] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:10.602881 2268 ts_tablet_manager.cc:1929] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:10.645679 2268 log.cc:1199] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/f88a5c0d0e2c436c8d8b6afc1f3c2edf
I20260505 07:54:10.645629 1172 ts_tablet_manager.cc:1403] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.138s user 0.004s sys 0.000s
I20260505 07:54:10.646433 2268 ts_tablet_manager.cc:1950] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:10.647241 1176 tablet_bootstrap.cc:492] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:10.647358 1176 ts_tablet_manager.cc:1403] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.000s
I20260505 07:54:10.648046 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:10.648689 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:10.649164 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:10.649462 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:10.649791 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]). Expected version 1 got 0
I20260505 07:54:10.650033 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:10.650319 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:10.650568 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:10.650794 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1001dd888ecd4166ba217506739f71e8 (table default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3]). Expected version 2 got 0
I20260505 07:54:10.650936 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 21480c42b73e4bf0ab2453b085a867c0 (table default.Table_925028e721e24ba68f31fe1040d4c2fd [id=0faa6676bab04919984b580299143271]). Expected version 1 got 0
I20260505 07:54:10.651088 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:10.651335 1353 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
W20260505 07:54:10.651911 1357 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 09019fdf60374b11bf3427f21a55b5e0 consensus state has no leader
I20260505 07:54:10.652112 1357 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 132) took 1423 ms. Trace:
I20260505 07:54:10.652225 1357 rpcz_store.cc:276] 0505 07:54:09.228539 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.354335 (+1125796us) service_pool.cc:224] Handling call
0505 07:54:10.355888 (+ 1553us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:10.356300 (+ 412us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:10.356665 (+ 365us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:10.356794 (+ 129us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:10.376245 (+ 19451us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:10.652072 (+275827us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.357918 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.358011 (+ 93us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.358018 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:10.358020 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:10.358114 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.358126 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.358131 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:10.358189 (+ 58us) tablet.cc:701] Row locks acquired
0505 07:54:10.358192 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:10.358225 (+ 33us) write_op.cc:270] Start()
0505 07:54:10.358257 (+ 32us) write_op.cc:276] Timestamp: P: 1777967650358190 usec, L: 0
0505 07:54:10.358261 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.358437 (+ 176us) log.cc:844] Serialized 405 byte log entry
0505 07:54:10.367990 (+ 9553us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.368113 (+ 123us) write_op.cc:301] APPLY: starting
0505 07:54:10.368129 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.368169 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.368207 (+ 38us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.368265 (+ 58us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.368304 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:10.368314 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:10.368398 (+ 84us) log.cc:844] Serialized 23 byte log entry
0505 07:54:10.368461 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.368480 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:10.368487 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:10.368567 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.368571 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":71,"apply.run_cpu_time_us":442,"apply.run_wall_time_us":442,"num_ops":1,"prepare.queue_time_us":99,"prepare.run_cpu_time_us":625,"prepare.run_wall_time_us":625,"raft.queue_time_us":391,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":218,"replication_time_us":9696}]]}
I20260505 07:54:10.653106 1357 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6bf0ac886e27494d867dd60c897ba88f" } modify_external_catalogs: true
I20260505 07:54:10.653290 1357 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6bf0ac886e27494d867dd60c897ba88f" } modify_external_catalogs: true
I20260505 07:54:10.655001 1351 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 120) took 2884 ms. Trace:
I20260505 07:54:10.655110 1351 rpcz_store.cc:276] 0505 07:54:07.770967 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.711164 (+1940197us) service_pool.cc:224] Handling call
0505 07:54:09.711480 (+ 316us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:09.711699 (+ 219us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:09.711843 (+ 144us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:09.711960 (+ 117us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:09.722451 (+ 10491us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:10.654956 (+932505us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.716032 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.716126 (+ 94us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.716133 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:09.716136 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:09.716219 (+ 83us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.716229 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.716233 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:09.716303 (+ 70us) tablet.cc:701] Row locks acquired
0505 07:54:09.716307 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:09.716338 (+ 31us) write_op.cc:270] Start()
0505 07:54:09.716363 (+ 25us) write_op.cc:276] Timestamp: P: 1777967649716315 usec, L: 0
0505 07:54:09.716367 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.716515 (+ 148us) log.cc:844] Serialized 405 byte log entry
0505 07:54:09.720334 (+ 3819us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.720430 (+ 96us) write_op.cc:301] APPLY: starting
0505 07:54:09.720444 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.720480 (+ 36us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.720514 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.720558 (+ 44us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.720587 (+ 29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:09.720595 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:09.720661 (+ 66us) log.cc:844] Serialized 23 byte log entry
0505 07:54:09.720713 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.720727 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:09.720734 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.720798 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.720802 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":53,"apply.run_cpu_time_us":361,"apply.run_wall_time_us":361,"num_ops":1,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":577,"prepare.run_wall_time_us":576,"raft.queue_time_us":56,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":312,"replication_time_us":3944,"spinlock_wait_cycles":11136}]]}
I20260505 07:54:10.655869 1351 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_dcd2d0ba71434f6bb31d92683f67408c" } modify_external_catalogs: true
I20260505 07:54:10.654927 1172 raft_consensus.cc:359] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.656030 1351 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_dcd2d0ba71434f6bb31d92683f67408c" } modify_external_catalogs: true
I20260505 07:54:10.656093 1172 raft_consensus.cc:385] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.656576 1172 raft_consensus.cc:740] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.657182 1349 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet df3a49b59ed6485f970074391f932482
I20260505 07:54:10.657606 1349 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 121) took 2886 ms. Trace:
I20260505 07:54:10.657363 1172 consensus_queue.cc:260] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.657703 1349 rpcz_store.cc:276] 0505 07:54:07.771074 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.714045 (+1942971us) service_pool.cc:224] Handling call
0505 07:54:09.714702 (+ 657us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:09.714856 (+ 154us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:09.714920 (+ 64us) catalog_manager.cc:2824] Locking tablets
0505 07:54:09.761325 (+ 46405us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:09.830842 (+ 69517us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:09.879192 (+ 48350us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:10.657120 (+777928us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:10.657571 (+ 451us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.804580 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.804694 (+ 114us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.804699 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:09.804701 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:09.804821 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.804830 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.804834 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.804950 (+ 116us) tablet.cc:701] Row locks acquired
0505 07:54:09.804953 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:09.804999 (+ 46us) write_op.cc:270] Start()
0505 07:54:09.805029 (+ 30us) write_op.cc:276] Timestamp: P: 1777967649804949 usec, L: 0
0505 07:54:09.805032 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.805200 (+ 168us) log.cc:844] Serialized 756 byte log entry
0505 07:54:09.828536 (+ 23336us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.829295 (+ 759us) write_op.cc:301] APPLY: starting
0505 07:54:09.829307 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.829344 (+ 37us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.829394 (+ 50us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.829435 (+ 41us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.829457 (+ 22us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:09.829463 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:09.829506 (+ 43us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.829538 (+ 32us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.829549 (+ 11us) write_op.cc:454] Released schema lock
0505 07:54:09.829553 (+ 4us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.829605 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.829608 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":723,"apply.run_cpu_time_us":307,"apply.run_wall_time_us":307,"mutex_wait_us":35313,"num_ops":2,"prepare.queue_time_us":42387,"prepare.run_cpu_time_us":679,"prepare.run_wall_time_us":679,"replication_time_us":23482}]]}
I20260505 07:54:10.658474 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_497d4c8424b841d38672247e2b4e33f2" } new_table_name: "default.Table_ba009e4831a44471ac8a72fffba6674a" modify_external_catalogs: true
I20260505 07:54:10.659720 1348 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet b256d072933b4566b5587224274deb66
I20260505 07:54:10.660322 1172 ts_tablet_manager.cc:1434] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.013s user 0.003s sys 0.000s
I20260505 07:54:10.661026 1348 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 122) took 2889 ms. Trace:
I20260505 07:54:10.661341 892 tablet_service.cc:1558] Processing DeleteTablet for tablet df3a49b59ed6485f970074391f932482 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.661110 1348 rpcz_store.cc:276] 0505 07:54:07.771199 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.716296 (+1945097us) service_pool.cc:224] Handling call
0505 07:54:09.716759 (+ 463us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:09.717090 (+ 331us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:09.717147 (+ 57us) catalog_manager.cc:2824] Locking tablets
0505 07:54:09.759904 (+ 42757us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:09.830301 (+ 70397us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:09.879133 (+ 48832us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:10.659676 (+780543us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:10.660141 (+ 465us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.803819 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.803934 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.803941 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:09.803944 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:09.804058 (+ 114us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.804068 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.804072 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:09.804191 (+ 119us) tablet.cc:701] Row locks acquired
0505 07:54:09.804194 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:09.804227 (+ 33us) write_op.cc:270] Start()
0505 07:54:09.804261 (+ 34us) write_op.cc:276] Timestamp: P: 1777967649804194 usec, L: 0
0505 07:54:09.804266 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.804465 (+ 199us) log.cc:844] Serialized 725 byte log entry
0505 07:54:09.828451 (+ 23986us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.828779 (+ 328us) write_op.cc:301] APPLY: starting
0505 07:54:09.828793 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.828842 (+ 49us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.828865 (+ 23us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.828935 (+ 70us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.828979 (+ 44us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:09.828990 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:09.829068 (+ 78us) log.cc:844] Serialized 29 byte log entry
0505 07:54:09.829132 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.829152 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:09.829160 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.829224 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.829228 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":276,"apply.run_cpu_time_us":440,"apply.run_wall_time_us":439,"mutex_wait_us":36856,"num_ops":2,"prepare.queue_time_us":43221,"prepare.run_cpu_time_us":719,"prepare.run_wall_time_us":719,"replication_time_us":24147}]]}
I20260505 07:54:10.661515 885 tablet_service.cc:1558] Processing DeleteTablet for tablet b256d072933b4566b5587224274deb66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.662420 2414 raft_consensus.cc:2749] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:10.662307 1172 tablet_bootstrap.cc:492] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:10.663058 757 tablet_service.cc:1558] Processing DeleteTablet for tablet df3a49b59ed6485f970074391f932482 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.663720 2089 raft_consensus.cc:493] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.663825 2089 raft_consensus.cc:515] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.664034 758 tablet_service.cc:1558] Processing DeleteTablet for tablet b256d072933b4566b5587224274deb66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.664319 2089 leader_election.cc:290] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.664374 1176 raft_consensus.cc:359] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.664525 1176 raft_consensus.cc:385] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.664573 1176 raft_consensus.cc:740] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.664733 1176 consensus_queue.cc:260] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.665175 1176 ts_tablet_manager.cc:1434] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260505 07:54:10.665491 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1602ccbae24544c4a6cf14ea073880b4" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.666288 2105 consensus_queue.cc:1048] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:10.666989 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 45259.8us
I20260505 07:54:10.668489 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet b256d072933b4566b5587224274deb66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:10.675750 1358 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d86214559d014f028acc5a9691efc9ca consensus state has no leader
I20260505 07:54:10.678846 1348 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_cc718440246d4ddfb49faf4602910188" } new_table_name: "default.Table_58eed87aa11e4e3fa7a7d18a135128a0" modify_external_catalogs: true
I20260505 07:54:10.680464 1172 tablet_bootstrap.cc:654] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.681789 2564 raft_consensus.cc:493] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.681928 2564 raft_consensus.cc:515] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.683050 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1602ccbae24544c4a6cf14ea073880b4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.683207 775 raft_consensus.cc:2468] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:10.683163 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d442119970a345f28de029eb2139f5c1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:10.683826 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423c22891bed462d89e9ec51a079f0f5" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.684108 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1602ccbae24544c4a6cf14ea073880b4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:10.684224 781 raft_consensus.cc:3060] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:10.684263 854 leader_election.cc:343] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.687054 781 raft_consensus.cc:2468] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:10.688844 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423c22891bed462d89e9ec51a079f0f5" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.688663 2564 leader_election.cc:290] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:10.689289 856 leader_election.cc:343] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.689393 856 leader_election.cc:304] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.693368 2564 raft_consensus.cc:2749] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.697314 2558 raft_consensus.cc:493] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.697436 2558 raft_consensus.cc:515] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.698027 2558 leader_election.cc:290] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.698678 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21480c42b73e4bf0ab2453b085a867c0" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.700425 1181 tablet_bootstrap.cc:492] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:10.701190 1172 tablet_bootstrap.cc:492] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:10.701422 1172 ts_tablet_manager.cc:1403] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.039s user 0.006s sys 0.000s
I20260505 07:54:10.704134 1172 raft_consensus.cc:359] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.704509 1172 raft_consensus.cc:385] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.704725 1172 raft_consensus.cc:740] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.705260 1181 ts_tablet_manager.cc:1403] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.138s user 0.004s sys 0.000s
I20260505 07:54:10.705209 1172 consensus_queue.cc:260] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.706904 1172 ts_tablet_manager.cc:1434] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260505 07:54:10.708029 1172 tablet_bootstrap.cc:492] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:10.701200 2307 raft_consensus.cc:493] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.715714 2307 raft_consensus.cc:515] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.716295 2307 leader_election.cc:290] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.716922 1172 tablet_bootstrap.cc:654] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.717001 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad8a79245ad47738d079752e51aed2b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.717303 912 raft_consensus.cc:2468] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:10.717751 991 leader_election.cc:304] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:10.718070 2307 raft_consensus.cc:2804] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:10.718139 2307 raft_consensus.cc:493] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:10.718195 2307 raft_consensus.cc:3060] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.721048 2307 raft_consensus.cc:515] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.721988 2307 leader_election.cc:290] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:10.723122 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad8a79245ad47738d079752e51aed2b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:10.723253 912 raft_consensus.cc:3060] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:10.725890 912 raft_consensus.cc:2468] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:10.726557 991 leader_election.cc:304] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:10.727092 2405 raft_consensus.cc:2804] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:10.727245 2405 raft_consensus.cc:697] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:10.727523 2405 consensus_queue.cc:237] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.712538 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 105) took 1635 ms. Trace:
I20260505 07:54:10.731060 2268 rpcz_store.cc:276] 0505 07:54:09.077213 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.078259 (+ 1046us) service_pool.cc:224] Handling call
0505 07:54:10.376860 (+1298601us) ts_tablet_manager.cc:1151] Deleting tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf
0505 07:54:10.376868 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.498035 (+121167us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.602829 (+104794us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.712523 (+109694us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1284573}
I20260505 07:54:10.732052 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 104) took 1654 ms. Trace:
I20260505 07:54:10.732583 2392 raft_consensus.cc:493] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.732760 2392 raft_consensus.cc:515] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.733016 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad8a79245ad47738d079752e51aed2b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.733389 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ad8a79245ad47738d079752e51aed2b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:10.733705 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6607b7428044e428af9d765c9c9b3ba" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:10.734205 854 leader_election.cc:343] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.734745 1358 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 123) took 2963 ms. Trace:
I20260505 07:54:10.716429 1181 raft_consensus.cc:359] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.734977 1181 raft_consensus.cc:385] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.734949 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6607b7428044e428af9d765c9c9b3ba" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:10.735056 1181 raft_consensus.cc:740] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.735239 1181 consensus_queue.cc:260] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.734879 1358 rpcz_store.cc:276] 0505 07:54:07.771305 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.725059 (+1953754us) service_pool.cc:224] Handling call
0505 07:54:09.725510 (+ 451us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:09.725759 (+ 249us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:09.725944 (+ 185us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:09.726012 (+ 68us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:09.738921 (+ 12909us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:10.734682 (+995761us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:09.726538 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:09.726623 (+ 85us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:09.726627 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:09.726629 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:09.726699 (+ 70us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:09.726708 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:09.726711 (+ 3us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:09.726754 (+ 43us) tablet.cc:701] Row locks acquired
0505 07:54:09.726756 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:09.726780 (+ 24us) write_op.cc:270] Start()
0505 07:54:09.726805 (+ 25us) write_op.cc:276] Timestamp: P: 1777967649726759 usec, L: 0
0505 07:54:09.726810 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:09.726979 (+ 169us) log.cc:844] Serialized 405 byte log entry
0505 07:54:09.729654 (+ 2675us) op_driver.cc:464] REPLICATION: finished
0505 07:54:09.729792 (+ 138us) write_op.cc:301] APPLY: starting
0505 07:54:09.729806 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:09.729840 (+ 34us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:09.729879 (+ 39us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:09.729922 (+ 43us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:09.729972 (+ 50us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:09.729982 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:09.730045 (+ 63us) log.cc:844] Serialized 23 byte log entry
0505 07:54:09.730093 (+ 48us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.730106 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:09.730111 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:09.730166 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:09.730169 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":94,"apply.run_cpu_time_us":368,"apply.run_wall_time_us":367,"num_ops":1,"prepare.queue_time_us":60,"prepare.run_cpu_time_us":542,"prepare.run_wall_time_us":541,"raft.queue_time_us":65,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":217,"replication_time_us":2811}]]}
W20260505 07:54:10.735630 856 leader_election.cc:343] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.735723 856 leader_election.cc:304] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.735874 1181 ts_tablet_manager.cc:1434] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.020s user 0.001s sys 0.000s
I20260505 07:54:10.736416 2505 raft_consensus.cc:2749] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.736555 2505 raft_consensus.cc:493] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:10.736598 1181 tablet_bootstrap.cc:492] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:10.737092 1351 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 17b4e886bebc4ab6a3c95290d18b4229
I20260505 07:54:10.736657 2505 raft_consensus.cc:515] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:10.737639 2505 leader_election.cc:290] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:10.738919 1181 tablet_bootstrap.cc:654] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.739215 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b57c1c618484d5d86a846fb982a2240" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.740428 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 17b4e886bebc4ab6a3c95290d18b4229 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:10.740655 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 64ca343bd32a4eeb80901bc3157b3b6e consensus state has no leader
I20260505 07:54:10.740708 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet df3a49b59ed6485f970074391f932482 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:10.740808 1349 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 135) took 1261 ms. Trace:
I20260505 07:54:10.741017 1172 tablet_bootstrap.cc:492] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:10.741252 1351 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 134) took 1468 ms. Trace:
I20260505 07:54:10.741268 1172 ts_tablet_manager.cc:1403] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.034s user 0.008s sys 0.000s
I20260505 07:54:10.741427 1351 rpcz_store.cc:276] 0505 07:54:09.272894 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.655726 (+1382832us) service_pool.cc:224] Handling call
0505 07:54:10.656119 (+ 393us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:10.656280 (+ 161us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:10.656331 (+ 51us) catalog_manager.cc:2824] Locking tablets
0505 07:54:10.656511 (+ 180us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:10.689617 (+ 33106us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:10.736972 (+ 47355us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:10.737040 (+ 68us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:10.741191 (+ 4151us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.662085 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.662184 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.662190 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:10.662193 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:10.662313 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.662340 (+ 27us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.662346 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:10.662453 (+ 107us) tablet.cc:701] Row locks acquired
0505 07:54:10.662456 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:10.662488 (+ 32us) write_op.cc:270] Start()
0505 07:54:10.662522 (+ 34us) write_op.cc:276] Timestamp: P: 1777967650662423 usec, L: 0
0505 07:54:10.662526 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.662672 (+ 146us) log.cc:844] Serialized 756 byte log entry
0505 07:54:10.688583 (+ 25911us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.688809 (+ 226us) write_op.cc:301] APPLY: starting
0505 07:54:10.688834 (+ 25us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.688868 (+ 34us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.688890 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.688962 (+ 72us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.688987 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:10.688994 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:10.689047 (+ 53us) log.cc:844] Serialized 29 byte log entry
0505 07:54:10.689100 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.689114 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:10.689120 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:10.689197 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.689201 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":180,"apply.run_cpu_time_us":366,"apply.run_wall_time_us":366,"num_ops":2,"prepare.queue_time_us":5014,"prepare.run_cpu_time_us":654,"prepare.run_wall_time_us":654,"replication_time_us":26030}]]}
I20260505 07:54:10.742419 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 17b4e886bebc4ab6a3c95290d18b4229 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.743072 1172 raft_consensus.cc:359] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.743412 1172 raft_consensus.cc:385] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.743553 1172 raft_consensus.cc:740] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.744161 1172 consensus_queue.cc:260] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.746847 1172 ts_tablet_manager.cc:1434] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
W20260505 07:54:10.748175 856 leader_election.cc:343] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 8b57c1c618484d5d86a846fb982a2240
I20260505 07:54:10.748287 856 leader_election.cc:304] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.748363 1172 tablet_bootstrap.cc:492] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:10.748603 2505 raft_consensus.cc:2749] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:10.749344 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1602ccbae24544c4a6cf14ea073880b4" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.749547 2392 leader_election.cc:290] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.749601 912 raft_consensus.cc:2393] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:10.750164 2392 raft_consensus.cc:493] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.750314 2392 raft_consensus.cc:515] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:10.750793 988 leader_election.cc:343] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.751205 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080b2984fcf2478a8def6540da63554d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:10.740943 1349 rpcz_store.cc:276] 0505 07:54:09.479446 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.658306 (+1178860us) service_pool.cc:224] Handling call
0505 07:54:10.658625 (+ 319us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:10.658826 (+ 201us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:10.659000 (+ 174us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:10.659098 (+ 98us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:10.692101 (+ 33003us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:10.740771 (+ 48670us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.662763 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.662854 (+ 91us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.662861 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:10.662863 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:10.669071 (+ 6208us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.669081 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.669087 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:10.679146 (+ 10059us) tablet.cc:701] Row locks acquired
0505 07:54:10.679149 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:10.679182 (+ 33us) write_op.cc:270] Start()
0505 07:54:10.679210 (+ 28us) write_op.cc:276] Timestamp: P: 1777967650679116 usec, L: 0
0505 07:54:10.679214 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.679364 (+ 150us) log.cc:844] Serialized 405 byte log entry
0505 07:54:10.688647 (+ 9283us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.689265 (+ 618us) write_op.cc:301] APPLY: starting
0505 07:54:10.689274 (+ 9us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.689298 (+ 24us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.689325 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.689354 (+ 29us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.689375 (+ 21us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:10.689382 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:10.689425 (+ 43us) log.cc:844] Serialized 23 byte log entry
0505 07:54:10.689457 (+ 32us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.689468 (+ 11us) write_op.cc:454] Released schema lock
0505 07:54:10.689472 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":585,"apply.run_cpu_time_us":261,"apply.run_wall_time_us":261,"num_ops":1,"prepare.queue_time_us":3304,"prepare.run_cpu_time_us":609,"prepare.run_wall_time_us":16666,"replication_time_us":9414,"threads_started":1}]]}
I20260505 07:54:10.751622 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080b2984fcf2478a8def6540da63554d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:10.751734 854 leader_election.cc:343] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:10.752177 856 leader_election.cc:343] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.752278 856 leader_election.cc:304] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.749120 1357 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 91d57f5bb6e847e6b099134d1382b730
I20260505 07:54:10.753289 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 91d57f5bb6e847e6b099134d1382b730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.753553 1357 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 133) took 1501 ms. Trace:
I20260505 07:54:10.753605 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21480c42b73e4bf0ab2453b085a867c0" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:10.754113 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 91d57f5bb6e847e6b099134d1382b730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.753684 1357 rpcz_store.cc:276] 0505 07:54:09.252277 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.652945 (+1400668us) service_pool.cc:224] Handling call
0505 07:54:10.653403 (+ 458us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:10.653574 (+ 171us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:10.653631 (+ 57us) catalog_manager.cc:2824] Locking tablets
0505 07:54:10.653786 (+ 155us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:10.684915 (+ 31129us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:10.749034 (+ 64119us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:10.749071 (+ 37us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:10.753500 (+ 4429us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.654544 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.654652 (+ 108us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.654661 (+ 9us) write_op.cc:435] Acquired schema lock
0505 07:54:10.654664 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:10.654770 (+ 106us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.654783 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.654787 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:10.654913 (+ 126us) tablet.cc:701] Row locks acquired
0505 07:54:10.654917 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:10.654954 (+ 37us) write_op.cc:270] Start()
0505 07:54:10.654993 (+ 39us) write_op.cc:276] Timestamp: P: 1777967650654888 usec, L: 0
0505 07:54:10.654996 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.655162 (+ 166us) log.cc:844] Serialized 725 byte log entry
0505 07:54:10.684203 (+ 29041us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.684326 (+ 123us) write_op.cc:301] APPLY: starting
0505 07:54:10.684339 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.684393 (+ 54us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.684422 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.684536 (+ 114us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.684571 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:10.684578 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:10.684643 (+ 65us) log.cc:844] Serialized 29 byte log entry
0505 07:54:10.684698 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.684714 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:10.684720 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:10.684783 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.684787 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":74,"apply.run_cpu_time_us":451,"apply.run_wall_time_us":450,"num_ops":2,"prepare.queue_time_us":53,"prepare.run_cpu_time_us":712,"prepare.run_wall_time_us":713,"raft.queue_time_us":5405,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":8588,"replication_time_us":29178}]]}
W20260505 07:54:10.755360 988 consensus_peers.cc:597] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:10.755926 899 tablet_service.cc:1558] Processing DeleteTablet for tablet f200abef13ac42698a9b96fc8af82ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.756590 887 tablet_service.cc:1558] Processing DeleteTablet for tablet ae829daad64545e687a9528e5c38b9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.756489 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 09b46ff0101a4e529077b9e30492ace7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.757226 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.757395 887 tablet_service.cc:1558] Processing DeleteTablet for tablet a9494c3b6a99487596a030856b371b68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6fe19306f7df49fdb771d21f6560e697) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.758011 899 tablet_service.cc:1558] Processing DeleteTablet for tablet e7887c3462f74740b314e3bec36e280d with delete_type TABLET_DATA_DELETED (replaced by tablet fba1d19f902a4a0a8b581135f6611129) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.757485 1172 tablet_bootstrap.cc:654] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.758244 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 02e06d8815034d91986cef12964860ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.758656 887 tablet_service.cc:1558] Processing DeleteTablet for tablet a9b7c6e9463342a5a73af12da4447f89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:10.758998 988 leader_election.cc:343] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: Tablet not RUNNING: SHUTDOWN
I20260505 07:54:10.759315 887 tablet_service.cc:1558] Processing DeleteTablet for tablet e82e9ae417784b7e83f4843b129d0580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.754595 2564 raft_consensus.cc:2749] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:10.759707 988 leader_election.cc:343] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: Tablet not RUNNING: SHUTDOWN
I20260505 07:54:10.760046 2392 leader_election.cc:290] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.760455 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (table default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c [id=8a910f6a553a47ea87ec170e376982d1]) successfully deleted
I20260505 07:54:10.760565 887 tablet_service.cc:1558] Processing DeleteTablet for tablet accb2dedc30e494ebd22c33594cee661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.762141 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 91d57f5bb6e847e6b099134d1382b730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:10.762964 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 17b4e886bebc4ab6a3c95290d18b4229 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:10.763453 2444 raft_consensus.cc:493] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.763630 2444 raft_consensus.cc:515] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.764470 2444 leader_election.cc:290] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.764757 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e06d8815034d91986cef12964860ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:10.765327 856 leader_election.cc:343] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.765633 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e06d8815034d91986cef12964860ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:10.766141 854 leader_election.cc:343] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.766283 854 leader_election.cc:304] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.768648 2392 raft_consensus.cc:2749] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.768904 2392 raft_consensus.cc:493] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.769062 2392 raft_consensus.cc:515] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.770357 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4dd3ece5cb34ceb82feeb67e75da0b5" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:10.770818 854 leader_election.cc:343] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.772387 2392 leader_election.cc:290] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.772447 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4dd3ece5cb34ceb82feeb67e75da0b5" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:10.772950 856 leader_election.cc:343] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.773211 856 leader_election.cc:304] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.775679 2392 raft_consensus.cc:2749] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.732321 2268 rpcz_store.cc:276] 0505 07:54:09.077107 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.078416 (+ 1309us) service_pool.cc:224] Handling call
0505 07:54:10.731794 (+1653378us) ts_tablet_manager.cc:1151] Deleting tablet 72da22d790624b07bfca6f6b6301c925
0505 07:54:10.731803 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.732003 (+ 200us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1639448}
W20260505 07:54:10.780814 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 72da22d790624b07bfca6f6b6301c925 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72da22d790624b07bfca6f6b6301c925 already in progress: creating tablet
I20260505 07:54:10.781417 1878 consensus_queue.cc:1048] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:10.782750 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 107) took 1242 ms. Trace:
I20260505 07:54:10.782909 2268 rpcz_store.cc:276] 0505 07:54:09.540156 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.542461 (+ 2305us) service_pool.cc:224] Handling call
0505 07:54:10.777754 (+1235293us) ts_tablet_manager.cc:1151] Deleting tablet abe5f6f0774545b296bf1b03a08e145d
0505 07:54:10.777764 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.782707 (+ 4943us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1235039}
I20260505 07:54:10.783350 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 108) took 1240 ms. Trace:
I20260505 07:54:10.783493 2268 rpcz_store.cc:276] 0505 07:54:09.542890 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.549974 (+ 7084us) service_pool.cc:224] Handling call
0505 07:54:10.783210 (+1233236us) ts_tablet_manager.cc:1151] Deleting tablet 2d86b8b4dadf4fd2b6409a8cb70e313e
0505 07:54:10.783218 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.783316 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1233063}
I20260505 07:54:10.784418 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 109) took 1233 ms. Trace:
I20260505 07:54:10.785476 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 127) took 1390 ms. Trace:
I20260505 07:54:10.785600 2447 rpcz_store.cc:276] 0505 07:54:09.394788 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.394853 (+ 65us) service_pool.cc:224] Handling call
0505 07:54:10.486250 (+1091397us) ts_tablet_manager.cc:1151] Deleting tablet 5ad8a79245ad47738d079752e51aed2b
0505 07:54:10.486258 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.495662 (+ 9404us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.512424 (+ 16762us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.785451 (+273027us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1091237}
I20260505 07:54:10.786085 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 128) took 1390 ms. Trace:
I20260505 07:54:10.786186 2447 rpcz_store.cc:276] 0505 07:54:09.395213 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.395261 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:10.785964 (+1390703us) ts_tablet_manager.cc:1151] Deleting tablet f2f5b72f58a24db0b58c15b47227bc60
0505 07:54:10.785976 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.786069 (+ 93us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1390553}
I20260505 07:54:10.786505 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 121) took 1393 ms. Trace:
I20260505 07:54:10.786578 2447 rpcz_store.cc:276] 0505 07:54:09.392692 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.395643 (+ 2951us) service_pool.cc:224] Handling call
0505 07:54:10.786412 (+1390769us) ts_tablet_manager.cc:1151] Deleting tablet d7e0c8ad716e443588d4ae3aa28ed4bf
0505 07:54:10.786421 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.786491 (+ 70us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1390540}
I20260505 07:54:10.786888 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 129) took 1391 ms. Trace:
I20260505 07:54:10.786963 2447 rpcz_store.cc:276] 0505 07:54:09.395604 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.395915 (+ 311us) service_pool.cc:224] Handling call
0505 07:54:10.786788 (+1390873us) ts_tablet_manager.cc:1151] Deleting tablet 02716f62aa354d8f839266661b7f05ea
0505 07:54:10.786797 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.786874 (+ 77us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1390713}
I20260505 07:54:10.787189 2447 tablet_replica.cc:333] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:10.787405 2447 raft_consensus.cc:2243] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.787600 2447 raft_consensus.cc:2272] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.788239 2447 ts_tablet_manager.cc:1916] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.790164 1176 tablet_bootstrap.cc:492] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:54:10.791234 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet abe5f6f0774545b296bf1b03a08e145d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abe5f6f0774545b296bf1b03a08e145d already in progress: creating tablet
W20260505 07:54:10.791718 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2d86b8b4dadf4fd2b6409a8cb70e313e already in progress: creating tablet
I20260505 07:54:10.793183 1176 tablet_bootstrap.cc:654] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:10.793907 718 consensus_peers.cc:597] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9044763bed1a41739ecce4def4cca1fe. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:10.795724 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 5ad8a79245ad47738d079752e51aed2b (table default.Table_ba3de8258d7e46c1ad3a509b8e00b806 [id=75d1988e3a9e4ff3a8c206a662f55131]) successfully deleted
W20260505 07:54:10.796042 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet f2f5b72f58a24db0b58c15b47227bc60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2f5b72f58a24db0b58c15b47227bc60 already in progress: creating tablet
W20260505 07:54:10.796718 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet d7e0c8ad716e443588d4ae3aa28ed4bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d7e0c8ad716e443588d4ae3aa28ed4bf already in progress: creating tablet
W20260505 07:54:10.796975 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 02716f62aa354d8f839266661b7f05ea because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 02716f62aa354d8f839266661b7f05ea already in progress: creating tablet
I20260505 07:54:10.797254 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 54e4f7f895d943639653c8c32724baf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.797369 1172 tablet_bootstrap.cc:492] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:10.797663 1172 ts_tablet_manager.cc:1403] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.050s user 0.005s sys 0.002s
I20260505 07:54:10.798669 718 leader_election.cc:304] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:10.798941 2558 raft_consensus.cc:2749] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.799670 718 leader_election.cc:304] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:10.801887 1172 raft_consensus.cc:359] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.802350 1172 raft_consensus.cc:385] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.802484 1172 raft_consensus.cc:740] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.800346 2447 ts_tablet_manager.cc:1929] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:10.805145 2447 log.cc:1199] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/b0b9e8eb767a4eceb583ff9b47000d58
I20260505 07:54:10.805315 2558 raft_consensus.cc:2749] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.805442 2447 ts_tablet_manager.cc:1950] T b0b9e8eb767a4eceb583ff9b47000d58 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:10.805747 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 130) took 1409 ms. Trace:
I20260505 07:54:10.805833 2447 rpcz_store.cc:276] 0505 07:54:09.396373 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.396424 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:10.787144 (+1390720us) ts_tablet_manager.cc:1151] Deleting tablet b0b9e8eb767a4eceb583ff9b47000d58
0505 07:54:10.787154 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.795211 (+ 8057us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.800306 (+ 5095us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.805715 (+ 5409us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1390561}
I20260505 07:54:10.806313 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 131) took 1378 ms. Trace:
I20260505 07:54:10.806514 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet b0b9e8eb767a4eceb583ff9b47000d58 (table default.Table_49c6ff5135b14b6b8437bb753e1b6342 [id=169b57b0e527456cb435bd97c47782bb]) successfully deleted
I20260505 07:54:10.805821 2268 rpcz_store.cc:276] 0505 07:54:09.550815 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.551910 (+ 1095us) service_pool.cc:224] Handling call
0505 07:54:10.784317 (+1232407us) ts_tablet_manager.cc:1151] Deleting tablet aab3850799254dda9716fd5bf9404159
0505 07:54:10.784325 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.784384 (+ 59us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1231890}
I20260505 07:54:10.807040 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 110) took 1254 ms. Trace:
I20260505 07:54:10.806392 2447 rpcz_store.cc:276] 0505 07:54:09.428387 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.428477 (+ 90us) service_pool.cc:224] Handling call
0505 07:54:10.806101 (+1377624us) ts_tablet_manager.cc:1151] Deleting tablet 080b2984fcf2478a8def6540da63554d
0505 07:54:10.806110 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.806298 (+ 188us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1377390}
I20260505 07:54:10.807190 2268 rpcz_store.cc:276] 0505 07:54:09.552599 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.553924 (+ 1325us) service_pool.cc:224] Handling call
0505 07:54:10.806905 (+1252981us) ts_tablet_manager.cc:1151] Deleting tablet d7e0c8ad716e443588d4ae3aa28ed4bf
0505 07:54:10.806915 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.807008 (+ 93us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1252833}
I20260505 07:54:10.807653 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 111) took 1253 ms. Trace:
W20260505 07:54:10.807631 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 080b2984fcf2478a8def6540da63554d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 080b2984fcf2478a8def6540da63554d already in progress: creating tablet
I20260505 07:54:10.807796 2268 rpcz_store.cc:276] 0505 07:54:09.554243 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.561562 (+ 7319us) service_pool.cc:224] Handling call
0505 07:54:10.807507 (+1245945us) ts_tablet_manager.cc:1151] Deleting tablet ba616702dc6040bb805bb3ffc8a8a871
0505 07:54:10.807515 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.807613 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1245792}
I20260505 07:54:10.808141 2268 tablet_replica.cc:333] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:10.808228 2447 tablet_replica.cc:333] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:10.808449 2268 raft_consensus.cc:2243] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.808660 2268 raft_consensus.cc:2272] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.809216 2268 ts_tablet_manager.cc:1916] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:10.809705 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet aab3850799254dda9716fd5bf9404159 because the tablet was not found. No further retry: Not found: Tablet not found: aab3850799254dda9716fd5bf9404159
W20260505 07:54:10.810097 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet d7e0c8ad716e443588d4ae3aa28ed4bf because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d7e0c8ad716e443588d4ae3aa28ed4bf already in progress: creating tablet
W20260505 07:54:10.810475 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet ba616702dc6040bb805bb3ffc8a8a871 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ba616702dc6040bb805bb3ffc8a8a871 already in progress: creating tablet
I20260505 07:54:10.811094 1176 tablet_bootstrap.cc:492] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:10.811214 1176 ts_tablet_manager.cc:1403] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.004s
I20260505 07:54:10.812011 1176 raft_consensus.cc:359] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.812188 1176 raft_consensus.cc:385] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.812250 1176 raft_consensus.cc:740] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.812668 1176 consensus_queue.cc:260] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.813164 1176 ts_tablet_manager.cc:1434] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:10.813406 1176 tablet_bootstrap.cc:492] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:10.814375 1181 tablet_bootstrap.cc:492] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:10.814913 1176 tablet_bootstrap.cc:654] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.820811 2447 raft_consensus.cc:2243] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.821110 2447 raft_consensus.cc:2272] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.823483 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 9044763bed1a41739ecce4def4cca1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.828773 2447 ts_tablet_manager.cc:1916] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.832114 894 tablet_service.cc:1558] Processing DeleteTablet for tablet a7d8265c05b74b28959d0de41fad1fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.802858 1172 consensus_queue.cc:260] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.843113 1181 ts_tablet_manager.cc:1403] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.107s user 0.003s sys 0.000s
I20260505 07:54:10.845577 1172 ts_tablet_manager.cc:1434] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.048s user 0.001s sys 0.002s
W20260505 07:54:10.849076 2257 log.cc:927] Time spent T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Append to log took a long time: real 0.065s user 0.003s sys 0.000s
I20260505 07:54:10.848948 1172 tablet_bootstrap.cc:492] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:10.852782 2447 ts_tablet_manager.cc:1929] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:10.854763 1181 raft_consensus.cc:359] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.856983 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_79892c60ca2c461087524d2724724ab6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:10.857599 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79892c60ca2c461087524d2724724ab6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:10.858970 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_83f3d8e2b2634dd587c1a08618c17b29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:10.861105 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_83f3d8e2b2634dd587c1a08618c17b29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:10.864926 1351 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_54fb239bdc8941e7bb95f042ae4ee1f7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:10.865923 1351 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_54fb239bdc8941e7bb95f042ae4ee1f7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:10.868183 894 tablet_service.cc:1558] Processing DeleteTablet for tablet e6a5f823db6143319d743d363a77ea47 with delete_type TABLET_DATA_DELETED (replaced by tablet 90612f0afb4949f491e625c61e40d247) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.869225 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.869377 1357 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 139) took 1214 ms. Trace:
I20260505 07:54:10.871658 894 tablet_service.cc:1558] Processing DeleteTablet for tablet e278b86e7b5d444ebad8d0f6921d4c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:10.872104 1348 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b0d966bcd7f1419cbd090afc185ae78a consensus state has no leader
I20260505 07:54:10.869475 1357 rpcz_store.cc:276] 0505 07:54:09.654168 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.754352 (+1100184us) service_pool.cc:224] Handling call
0505 07:54:10.856070 (+101718us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d6b13 00007fd65a46a7c6 00007fd65a46a4f5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:10.857703 (+ 1633us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:10.858180 (+ 477us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:10.868070 (+ 9890us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:10.868159 (+ 89us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:10.868237 (+ 78us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.860611 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.861219 (+ 608us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.861225 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:10.861227 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:10.861301 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.861310 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.861314 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:10.861380 (+ 66us) tablet.cc:701] Row locks acquired
0505 07:54:10.861382 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:10.861412 (+ 30us) write_op.cc:270] Start()
0505 07:54:10.861440 (+ 28us) write_op.cc:276] Timestamp: P: 1777967650861381 usec, L: 0
0505 07:54:10.861443 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.861602 (+ 159us) log.cc:844] Serialized 433 byte log entry
0505 07:54:10.867128 (+ 5526us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.867231 (+ 103us) write_op.cc:301] APPLY: starting
0505 07:54:10.867244 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.867289 (+ 45us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.867314 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.867391 (+ 77us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.867431 (+ 40us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:10.867441 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:10.867523 (+ 82us) log.cc:844] Serialized 29 byte log entry
0505 07:54:10.867576 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.867613 (+ 37us) write_op.cc:454] Released schema lock
0505 07:54:10.867620 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:10.867697 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.867703 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":233252736,"child_traces":[["op",{"apply.queue_time_us":61,"apply.run_cpu_time_us":452,"apply.run_wall_time_us":451,"num_ops":2,"prepare.queue_time_us":1853,"prepare.run_cpu_time_us":1078,"prepare.run_wall_time_us":1542,"replication_time_us":5656}]]}
I20260505 07:54:10.878882 2447 log.cc:1199] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/d622612bdc404727a27b83769bf39738
I20260505 07:54:10.879858 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_80c5d2cc30df4219ac6df63476fe3e14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:10.884639 1181 raft_consensus.cc:385] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.884805 1181 raft_consensus.cc:740] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
W20260505 07:54:10.884953 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_80c5d2cc30df4219ac6df63476fe3e14 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:10.885185 1181 consensus_queue.cc:260] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.852113 2598 tablet.cc:1724] T 42360f8ee4ff47c2af8d287f792207de P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:10.895094 2596 tablet.cc:1724] T 42360f8ee4ff47c2af8d287f792207de P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:10.895556 2447 ts_tablet_manager.cc:1950] T d622612bdc404727a27b83769bf39738 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:10.904292 1176 tablet_bootstrap.cc:492] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:10.904573 1176 ts_tablet_manager.cc:1403] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.091s user 0.004s sys 0.003s
I20260505 07:54:10.906114 1176 raft_consensus.cc:359] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.906483 1176 raft_consensus.cc:385] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.906617 1176 raft_consensus.cc:740] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.906989 1176 consensus_queue.cc:260] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.908771 1181 ts_tablet_manager.cc:1434] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.055s user 0.001s sys 0.001s
I20260505 07:54:10.909291 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 132) took 1480 ms. Trace:
I20260505 07:54:10.910686 1348 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 136) took 1427 ms. Trace:
I20260505 07:54:10.910795 1348 rpcz_store.cc:276] 0505 07:54:09.482962 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.661622 (+1178660us) service_pool.cc:224] Handling call
0505 07:54:10.741677 (+ 80055us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:10.741876 (+ 199us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:10.742039 (+ 163us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:10.742116 (+ 77us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:10.870341 (+128225us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:10.910612 (+ 40271us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.742626 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.742752 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.742757 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:10.742759 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:10.742889 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.742901 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.742905 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:10.742960 (+ 55us) tablet.cc:701] Row locks acquired
0505 07:54:10.742962 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:10.742988 (+ 26us) write_op.cc:270] Start()
0505 07:54:10.743020 (+ 32us) write_op.cc:276] Timestamp: P: 1777967650742970 usec, L: 0
0505 07:54:10.743025 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.855569 (+112544us) spinlock_profiling.cc:243] Waited 7.97 ms on lock 0x6160000430b8. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65ba29775 00007fd65bb0e51b 00007fd65bae9518 00007fd65bae8dc5 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f
0505 07:54:10.855787 (+ 218us) log.cc:844] Serialized 405 byte log entry
0505 07:54:10.855942 (+ 155us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65bae8f22 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac
0505 07:54:10.860162 (+ 4220us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.862754 (+ 2592us) write_op.cc:301] APPLY: starting
0505 07:54:10.862769 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.868409 (+ 5640us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.868414 (+ 5us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.868499 (+ 85us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.868565 (+ 66us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:10.868573 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:10.870059 (+ 1486us) log.cc:844] Serialized 23 byte log entry
0505 07:54:10.870262 (+ 203us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.870274 (+ 12us) write_op.cc:454] Released schema lock
0505 07:54:10.870280 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:10.870686 (+ 406us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.870691 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8296,"child_traces":[["op",{"apply.queue_time_us":537,"apply.run_cpu_time_us":520,"apply.run_wall_time_us":7970,"num_ops":1,"prepare.queue_time_us":47,"prepare.run_cpu_time_us":2395,"prepare.run_wall_time_us":113377,"raft.queue_time_us":392,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":396,"replication_time_us":117091,"spinlock_wait_cycles":258091008}]]}
W20260505 07:54:10.911990 1315 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet b0d966bcd7f1419cbd090afc185ae78a (table default.Table_58eed87aa11e4e3fa7a7d18a135128a0 [id=3ae3bc457dcb4db1b5f4360cb9e5c81c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b0d966bcd7f1419cbd090afc185ae78a consensus state has no leader
I20260505 07:54:10.912045 1348 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_865701615b5f4c06884f469ea75d36f8" } modify_external_catalogs: true
I20260505 07:54:10.912214 1348 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_865701615b5f4c06884f469ea75d36f8" } modify_external_catalogs: true
I20260505 07:54:10.910696 1349 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 138) took 1256 ms. Trace:
I20260505 07:54:10.913378 1349 rpcz_store.cc:276] 0505 07:54:09.654064 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.752500 (+1098436us) service_pool.cc:224] Handling call
0505 07:54:10.858889 (+106389us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d6b13 00007fd65a46a7c6 00007fd65a46a4f5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:10.861205 (+ 2316us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:10.861804 (+ 599us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:10.873801 (+ 11997us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:10.910556 (+ 36755us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:10.910650 (+ 94us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.865411 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.865501 (+ 90us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.865505 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:10.865507 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:10.865575 (+ 68us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.865584 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.865588 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:10.865665 (+ 77us) tablet.cc:701] Row locks acquired
0505 07:54:10.865667 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:10.865693 (+ 26us) write_op.cc:270] Start()
0505 07:54:10.865724 (+ 31us) write_op.cc:276] Timestamp: P: 1777967650865662 usec, L: 0
0505 07:54:10.865728 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.865870 (+ 142us) log.cc:844] Serialized 433 byte log entry
0505 07:54:10.870465 (+ 4595us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.870592 (+ 127us) write_op.cc:301] APPLY: starting
0505 07:54:10.870606 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.870636 (+ 30us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.871529 (+ 893us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.871671 (+ 142us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.871718 (+ 47us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:10.871726 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:10.871806 (+ 80us) log.cc:844] Serialized 29 byte log entry
0505 07:54:10.871888 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.871907 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:10.871913 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:10.871979 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.871983 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":238979840,"child_traces":[["op",{"apply.queue_time_us":75,"apply.run_cpu_time_us":539,"apply.run_wall_time_us":1381,"num_ops":2,"prepare.queue_time_us":40,"prepare.run_cpu_time_us":520,"prepare.run_wall_time_us":519,"replication_time_us":4713}]]}
I20260505 07:54:10.914196 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_42065f531a0b4382bfdd52f1b70ebc9d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:10.914700 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_42065f531a0b4382bfdd52f1b70ebc9d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:10.915513 988 consensus_peers.cc:597] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:10.916395 1181 tablet_bootstrap.cc:492] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:10.909439 2447 rpcz_store.cc:276] 0505 07:54:09.429136 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.429215 (+ 79us) service_pool.cc:224] Handling call
0505 07:54:10.808193 (+1378978us) ts_tablet_manager.cc:1151] Deleting tablet d622612bdc404727a27b83769bf39738
0505 07:54:10.808201 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.847692 (+ 39491us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.852722 (+ 5030us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.909270 (+ 56548us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12382,"tablet-delete.queue_time_us":1378777}
I20260505 07:54:10.918437 1181 tablet_bootstrap.cc:654] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.924060 894 tablet_service.cc:1558] Processing DeleteTablet for tablet b12d2d2b866348d781209fa787da7d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.924247 2444 raft_consensus.cc:493] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.924424 2444 raft_consensus.cc:515] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.924868 1354 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_7abf5cf3a7424664853b584a24a36437 [id=0b202734a7744102a98cc55ac980603b] alter complete (version 3)
I20260505 07:54:10.925449 2444 leader_election.cc:290] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.926208 1352 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f] alter complete (version 1)
I20260505 07:54:10.932612 2444 raft_consensus.cc:493] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:10.932813 2444 raft_consensus.cc:515] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.933549 1172 tablet_bootstrap.cc:654] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.940699 2444 leader_election.cc:290] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:10.942183 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 06904a54b1c5463da31af0bc00fb606f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.942512 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 26f3544e74d941d286a2a0317b941ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.942793 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 4662d4a3d88d4c2883a35048d6e11eb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.943349 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.944564 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429ccdb56f0b456483cf1debb5de0beb" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:10.945176 856 leader_election.cc:343] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.945773 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.947688 894 tablet_service.cc:1558] Processing DeleteTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e08cf03ea0b495a94c6566ea6c8caad) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:10.940450 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429ccdb56f0b456483cf1debb5de0beb" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:10.949026 854 leader_election.cc:343] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.949012 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet d622612bdc404727a27b83769bf39738 (table default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]) successfully deleted
I20260505 07:54:10.949110 854 leader_election.cc:304] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.949295 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 133) took 1517 ms. Trace:
I20260505 07:54:10.949666 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3462dcc6391741e59dcb5cd08527e50f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:10.950196 856 leader_election.cc:343] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.949472 2447 rpcz_store.cc:276] 0505 07:54:09.431871 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.431933 (+ 62us) service_pool.cc:224] Handling call
0505 07:54:10.948738 (+1516805us) ts_tablet_manager.cc:1151] Deleting tablet e010f23c4df647ab863e03c3cfed7898
0505 07:54:10.948780 (+ 42us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.949275 (+ 495us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1516641}
I20260505 07:54:10.950555 2444 raft_consensus.cc:2749] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:10.956213 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e010f23c4df647ab863e03c3cfed7898 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e010f23c4df647ab863e03c3cfed7898 already in progress: creating tablet
I20260505 07:54:10.956692 2447 tablet_replica.cc:333] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:10.957129 2447 raft_consensus.cc:2243] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.957476 2447 raft_consensus.cc:2272] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.958607 2447 ts_tablet_manager.cc:1916] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.954254 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3462dcc6391741e59dcb5cd08527e50f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:10.968405 854 leader_election.cc:343] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:10.968516 854 leader_election.cc:304] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:10.948431 1176 ts_tablet_manager.cc:1434] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.044s user 0.001s sys 0.001s
I20260505 07:54:10.969810 1176 tablet_bootstrap.cc:492] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:10.971184 1352 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_efa75c9c9cb34200856f6e2816aa4deb [id=37397b87cfe445f2adfb7eb0af27454b] alter complete (version 2)
I20260505 07:54:10.972644 1176 tablet_bootstrap.cc:654] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:10.974143 2444 raft_consensus.cc:2749] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:10.975015 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.975204 751 tablet_service.cc:1558] Processing DeleteTablet for tablet a9494c3b6a99487596a030856b371b68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6fe19306f7df49fdb771d21f6560e697) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.976276 751 tablet_service.cc:1558] Processing DeleteTablet for tablet ae829daad64545e687a9528e5c38b9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.976440 758 tablet_service.cc:1558] Processing DeleteTablet for tablet a9b7c6e9463342a5a73af12da4447f89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.976619 749 tablet_service.cc:1558] Processing DeleteTablet for tablet 02e06d8815034d91986cef12964860ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.976689 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 with delete_type TABLET_DATA_DELETED (replaced by tablet 66082a7a75e345639e7a268727922fbb) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.976789 751 tablet_service.cc:1558] Processing DeleteTablet for tablet e82e9ae417784b7e83f4843b129d0580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.976922 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.977380 758 tablet_service.cc:1558] Processing DeleteTablet for tablet aab3850799254dda9716fd5bf9404159 with delete_type TABLET_DATA_DELETED (replaced by tablet 5006965919824adbbb23d744fb9ff335) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.977530 751 tablet_service.cc:1558] Processing DeleteTablet for tablet abe5f6f0774545b296bf1b03a08e145d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:10.979298 2276 ts_tablet_manager.cc:1929] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:10.979806 2276 log.cc:1199] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/bde59d52fd234343b1ca13a7d4aae6bf
I20260505 07:54:10.979636 2599 tablet.cc:1724] T 42360f8ee4ff47c2af8d287f792207de P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:10.991307 2276 ts_tablet_manager.cc:1950] T bde59d52fd234343b1ca13a7d4aae6bf P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:10.992065 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 67) took 3343 ms. Trace:
I20260505 07:54:10.992936 2276 rpcz_store.cc:276] 0505 07:54:07.648655 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.649854 (+ 1199us) service_pool.cc:224] Handling call
0505 07:54:10.277667 (+2627813us) ts_tablet_manager.cc:1151] Deleting tablet bde59d52fd234343b1ca13a7d4aae6bf
0505 07:54:10.277678 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.303803 (+ 26125us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.979227 (+675424us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:10.992048 (+ 12821us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2627668}
I20260505 07:54:10.993474 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet bde59d52fd234343b1ca13a7d4aae6bf (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]) successfully deleted
I20260505 07:54:10.994580 1176 tablet_bootstrap.cc:492] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:10.994786 2276 tablet_replica.cc:333] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:10.994861 1176 ts_tablet_manager.cc:1403] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.004s
I20260505 07:54:10.995339 2276 raft_consensus.cc:2243] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:10.996385 1176 raft_consensus.cc:359] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.996685 1176 raft_consensus.cc:385] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:10.996848 2276 raft_consensus.cc:2272] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:10.996814 1176 raft_consensus.cc:740] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:10.997152 1176 consensus_queue.cc:260] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:10.998625 1348 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 880e326cbea74233a0ec5abb31f5b0d7
I20260505 07:54:10.998914 1349 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 142) took 1083 ms. Trace:
I20260505 07:54:10.999738 2276 ts_tablet_manager.cc:1916] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:10.999400 1349 rpcz_store.cc:276] 0505 07:54:09.915570 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.914043 (+998473us) service_pool.cc:224] Handling call
0505 07:54:10.914796 (+ 753us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:10.915278 (+ 482us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:10.944297 (+ 29019us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:10.998773 (+ 54476us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:10.998859 (+ 86us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.915812 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.915918 (+ 106us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.915923 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:10.915925 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:10.916002 (+ 77us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.916011 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.916015 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:10.916082 (+ 67us) tablet.cc:701] Row locks acquired
0505 07:54:10.916084 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:10.916108 (+ 24us) write_op.cc:270] Start()
0505 07:54:10.916130 (+ 22us) write_op.cc:276] Timestamp: P: 1777967650916073 usec, L: 0
0505 07:54:10.916133 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.916259 (+ 126us) log.cc:844] Serialized 433 byte log entry
0505 07:54:10.943559 (+ 27300us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.943671 (+ 112us) write_op.cc:301] APPLY: starting
0505 07:54:10.943682 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.943711 (+ 29us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.943741 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.943826 (+ 85us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.943869 (+ 43us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:10.943879 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:10.943938 (+ 59us) log.cc:844] Serialized 29 byte log entry
0505 07:54:10.943984 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.943997 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:10.944003 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":66,"apply.run_cpu_time_us":382,"apply.run_wall_time_us":382,"num_ops":2,"prepare.queue_time_us":53,"prepare.run_cpu_time_us":508,"prepare.run_wall_time_us":509,"replication_time_us":27405}]]}
I20260505 07:54:11.000461 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_2a1375626b524e7fa087a6db2c2783f8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:11.001827 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2a1375626b524e7fa087a6db2c2783f8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:11.002591 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 880e326cbea74233a0ec5abb31f5b0d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.002283 1348 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 141) took 1135 ms. Trace:
I20260505 07:54:11.003091 1357 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 140) took 1319 ms. Trace:
I20260505 07:54:11.003203 1357 rpcz_store.cc:276] 0505 07:54:09.683821 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.874682 (+1190861us) service_pool.cc:224] Handling call
0505 07:54:10.910710 (+ 36028us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:10.911484 (+ 774us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:10.942676 (+ 31192us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:10.998463 (+ 55787us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:11.003022 (+ 4559us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.912568 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.912662 (+ 94us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.912667 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:10.912669 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:10.912756 (+ 87us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.912765 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.912768 (+ 3us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:10.912835 (+ 67us) tablet.cc:701] Row locks acquired
0505 07:54:10.912837 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:10.912865 (+ 28us) write_op.cc:270] Start()
0505 07:54:10.912900 (+ 35us) write_op.cc:276] Timestamp: P: 1777967650912829 usec, L: 0
0505 07:54:10.912905 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.913052 (+ 147us) log.cc:844] Serialized 433 byte log entry
0505 07:54:10.942141 (+ 29089us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.942215 (+ 74us) write_op.cc:301] APPLY: starting
0505 07:54:10.942227 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.942278 (+ 51us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.942313 (+ 35us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.942400 (+ 87us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.942430 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:10.942439 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:10.942499 (+ 60us) log.cc:844] Serialized 29 byte log entry
0505 07:54:10.942563 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.942580 (+ 17us) write_op.cc:454] Released schema lock
0505 07:54:10.942586 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":27,"apply.run_cpu_time_us":438,"apply.run_wall_time_us":438,"num_ops":2,"prepare.queue_time_us":41,"prepare.run_cpu_time_us":544,"prepare.run_wall_time_us":544,"replication_time_us":29209}]]}
I20260505 07:54:11.002761 1348 rpcz_store.cc:276] 0505 07:54:09.866817 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.911829 (+1045012us) service_pool.cc:224] Handling call
0505 07:54:10.912307 (+ 478us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:10.912447 (+ 140us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:10.912498 (+ 51us) catalog_manager.cc:2824] Locking tablets
0505 07:54:10.912645 (+ 147us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:10.944396 (+ 31751us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:10.998512 (+ 54116us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:10.998558 (+ 46us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:11.002242 (+ 3684us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.913503 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.913611 (+ 108us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.913615 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:10.913617 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:10.913716 (+ 99us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.913724 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.913727 (+ 3us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:10.913816 (+ 89us) tablet.cc:701] Row locks acquired
0505 07:54:10.913819 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:10.913857 (+ 38us) write_op.cc:270] Start()
0505 07:54:10.913881 (+ 24us) write_op.cc:276] Timestamp: P: 1777967650913821 usec, L: 0
0505 07:54:10.913884 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.914025 (+ 141us) log.cc:844] Serialized 725 byte log entry
0505 07:54:10.942915 (+ 28890us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.942989 (+ 74us) write_op.cc:301] APPLY: starting
0505 07:54:10.943002 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.943032 (+ 30us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.943053 (+ 21us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.943111 (+ 58us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.943138 (+ 27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:10.943148 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:10.943200 (+ 52us) log.cc:844] Serialized 29 byte log entry
0505 07:54:10.943250 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.943263 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:10.943269 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":32,"apply.run_cpu_time_us":333,"apply.run_wall_time_us":332,"num_ops":2,"prepare.queue_time_us":265,"prepare.run_cpu_time_us":579,"prepare.run_wall_time_us":579,"replication_time_us":28990}]]}
I20260505 07:54:11.004261 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_d798bdca20804d6eb6aa52b61c2b69e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:11.004771 1348 master_service.cc:954] ReplaceTablet: received request to replace tablet f842d8ac6b2142ffa69945c6087be34b from {username='slave'} at 127.0.0.1:46452
W20260505 07:54:11.004971 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d798bdca20804d6eb6aa52b61c2b69e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:11.001314 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 880e326cbea74233a0ec5abb31f5b0d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:11.007097 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 472441bd29704be59aabb0f7a4964bb0 with delete_type TABLET_DATA_DELETED (replaced by tablet e7887c3462f74740b314e3bec36e280d) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.002831 1172 tablet_bootstrap.cc:492] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:11.009332 1172 ts_tablet_manager.cc:1403] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.161s user 0.005s sys 0.006s
I20260505 07:54:11.009872 1351 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 137) took 1476 ms. Trace:
I20260505 07:54:11.009970 1351 rpcz_store.cc:276] 0505 07:54:09.533765 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.749314 (+1215549us) service_pool.cc:224] Handling call
0505 07:54:10.862546 (+113232us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d6b13 00007fd65a46a7c6 00007fd65a46a4f5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:10.866128 (+ 3582us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:10.867046 (+ 918us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:10.922580 (+ 55534us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:10.998404 (+ 75824us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:11.009821 (+ 11417us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.872201 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.872318 (+ 117us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.872324 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:10.872328 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:10.872421 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.872433 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.872439 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:10.872521 (+ 82us) tablet.cc:701] Row locks acquired
0505 07:54:10.872525 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:10.872575 (+ 50us) write_op.cc:270] Start()
0505 07:54:10.872603 (+ 28us) write_op.cc:276] Timestamp: P: 1777967650872522 usec, L: 0
0505 07:54:10.872608 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.872754 (+ 146us) log.cc:844] Serialized 433 byte log entry
0505 07:54:10.922008 (+ 49254us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.922090 (+ 82us) write_op.cc:301] APPLY: starting
0505 07:54:10.922106 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.922148 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.922179 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.922283 (+ 104us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.922323 (+ 40us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:10.922332 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:10.922399 (+ 67us) log.cc:844] Serialized 29 byte log entry
0505 07:54:10.922455 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.922473 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:10.922481 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":245423488,"child_traces":[["op",{"apply.queue_time_us":34,"apply.run_cpu_time_us":460,"apply.run_wall_time_us":460,"num_ops":2,"prepare.queue_time_us":38,"prepare.run_cpu_time_us":636,"prepare.run_wall_time_us":636,"replication_time_us":49374}]]}
I20260505 07:54:11.010700 1351 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_7d9b2722b99c42b39ed977c61d37623b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:11.011148 1351 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7d9b2722b99c42b39ed977c61d37623b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:11.011093 1172 raft_consensus.cc:359] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.011420 1172 raft_consensus.cc:385] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.011521 1172 raft_consensus.cc:740] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.011967 1172 consensus_queue.cc:260] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.013113 1172 ts_tablet_manager.cc:1434] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:11.013764 1172 tablet_bootstrap.cc:492] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:11.022048 854 consensus_peers.cc:597] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0b9e8eb767a4eceb583ff9b47000d58. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.023218 2447 ts_tablet_manager.cc:1929] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:11.023322 2447 log.cc:1199] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/1602ccbae24544c4a6cf14ea073880b4
I20260505 07:54:11.023653 2447 ts_tablet_manager.cc:1950] T 1602ccbae24544c4a6cf14ea073880b4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:11.023856 2276 ts_tablet_manager.cc:1929] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.024479 2276 log.cc:1199] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/17327cee69a24e0cb493ff55f948bf6c
I20260505 07:54:11.024765 1357 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 144) took 1094 ms. Trace:
I20260505 07:54:11.024943 2276 ts_tablet_manager.cc:1950] T 17327cee69a24e0cb493ff55f948bf6c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:11.024871 1357 rpcz_store.cc:276] 0505 07:54:09.930152 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.003801 (+1073649us) service_pool.cc:224] Handling call
0505 07:54:11.005255 (+ 1454us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:11.008538 (+ 3283us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:11.024477 (+ 15939us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:11.024610 (+ 133us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:11.024703 (+ 93us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:11.009005 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.009084 (+ 79us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.009088 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:11.009090 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:11.009156 (+ 66us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.009165 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.009169 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:11.009246 (+ 77us) tablet.cc:701] Row locks acquired
0505 07:54:11.009248 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:11.009277 (+ 29us) write_op.cc:270] Start()
0505 07:54:11.009298 (+ 21us) write_op.cc:276] Timestamp: P: 1777967651009233 usec, L: 0
0505 07:54:11.009301 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.009425 (+ 124us) log.cc:844] Serialized 433 byte log entry
0505 07:54:11.023453 (+ 14028us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.024049 (+ 596us) write_op.cc:301] APPLY: starting
0505 07:54:11.024058 (+ 9us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.024089 (+ 31us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.024118 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.024215 (+ 97us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.024243 (+ 28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:11.024249 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:11.024318 (+ 69us) log.cc:844] Serialized 29 byte log entry
0505 07:54:11.024355 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.024367 (+ 12us) write_op.cc:454] Released schema lock
0505 07:54:11.024374 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":570,"apply.run_cpu_time_us":382,"apply.run_wall_time_us":382,"num_ops":2,"prepare.queue_time_us":40,"prepare.run_cpu_time_us":473,"prepare.run_wall_time_us":472,"replication_time_us":14138}]]}
I20260505 07:54:11.025626 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_91e9ab2f967244e6a89f8c76b0f58e93"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:11.026094 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91e9ab2f967244e6a89f8c76b0f58e93 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:11.026631 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 66) took 3378 ms. Trace:
I20260505 07:54:11.026750 2276 rpcz_store.cc:276] 0505 07:54:07.648572 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.650106 (+ 1534us) service_pool.cc:224] Handling call
0505 07:54:10.993781 (+3343675us) ts_tablet_manager.cc:1151] Deleting tablet 17327cee69a24e0cb493ff55f948bf6c
0505 07:54:10.994720 (+ 939us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.015707 (+ 20987us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.023802 (+ 8095us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.026613 (+ 2811us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3343513}
I20260505 07:54:11.027168 1354 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_98b055003e064e57b2ec488952e58844 [id=8f533b7aa2374c439f7234a373af31bb] alter complete (version 4)
I20260505 07:54:11.027246 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 17327cee69a24e0cb493ff55f948bf6c (table default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]) successfully deleted
I20260505 07:54:11.028060 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 64) took 3379 ms. Trace:
I20260505 07:54:11.028203 2276 rpcz_store.cc:276] 0505 07:54:07.648364 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.650342 (+ 1978us) service_pool.cc:224] Handling call
0505 07:54:11.027944 (+3377602us) ts_tablet_manager.cc:1151] Deleting tablet 840eebf9e7404bf7807b7e51996957e5
0505 07:54:11.027958 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.028040 (+ 82us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3377425}
W20260505 07:54:11.028827 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 840eebf9e7404bf7807b7e51996957e5 because the tablet was not found. No further retry: Not found: Tablet not found: 840eebf9e7404bf7807b7e51996957e5
I20260505 07:54:11.029186 2276 tablet_replica.cc:333] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:11.029412 2276 raft_consensus.cc:2243] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:11.029573 1349 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 143) took 1112 ms. Trace:
I20260505 07:54:11.029737 2276 raft_consensus.cc:2272] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.030423 2276 ts_tablet_manager.cc:1916] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.030210 1351 master_service.cc:954] ReplaceTablet: received request to replace tablet 0baf83b155494fddb3c25144608706f7 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:11.031378 1351 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_3ba75b2103cc434f8eaa19af274c5ff9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:11.031872 1351 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3ba75b2103cc434f8eaa19af274c5ff9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:11.030901 1349 rpcz_store.cc:276] 0505 07:54:09.917422 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.000264 (+1082842us) service_pool.cc:224] Handling call
0505 07:54:11.002410 (+ 2146us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:11.003394 (+ 984us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:11.029335 (+ 25941us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:11.029442 (+ 107us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:11.029532 (+ 90us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:11.005349 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.005442 (+ 93us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.005447 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:11.005450 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:11.005541 (+ 91us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.005556 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.005573 (+ 17us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:11.005647 (+ 74us) tablet.cc:701] Row locks acquired
0505 07:54:11.005649 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:11.005685 (+ 36us) write_op.cc:270] Start()
0505 07:54:11.005719 (+ 34us) write_op.cc:276] Timestamp: P: 1777967651005637 usec, L: 0
0505 07:54:11.005722 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.005905 (+ 183us) log.cc:844] Serialized 433 byte log entry
0505 07:54:11.017310 (+ 11405us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.017406 (+ 96us) write_op.cc:301] APPLY: starting
0505 07:54:11.017420 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.017471 (+ 51us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.017521 (+ 50us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.017646 (+ 125us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.017684 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:11.017692 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:11.017769 (+ 77us) log.cc:844] Serialized 29 byte log entry
0505 07:54:11.017839 (+ 70us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.017859 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:11.017867 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":45,"apply.run_cpu_time_us":549,"apply.run_wall_time_us":549,"num_ops":2,"prepare.queue_time_us":39,"prepare.run_cpu_time_us":653,"prepare.run_wall_time_us":653,"raft.queue_time_us":401,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":250,"replication_time_us":11550}]]}
I20260505 07:54:11.033699 1176 ts_tablet_manager.cc:1434] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.039s user 0.001s sys 0.001s
I20260505 07:54:11.036106 2405 raft_consensus.cc:493] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.036260 2405 raft_consensus.cc:515] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.037676 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 880e326cbea74233a0ec5abb31f5b0d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.040566 1349 master_service.cc:954] ReplaceTablet: received request to replace tablet 297537a92684425d8508f7b3b4c9d379 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:11.042006 1348 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet f842d8ac6b2142ffa69945c6087be34b
I20260505 07:54:11.042816 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_4ec0d26b7f5e4c25ac10858c5718448a" } new_table_name: "default.Table_23cfcb1d56bc4e72af90bc263a464bf2" modify_external_catalogs: true
I20260505 07:54:11.043129 751 tablet_service.cc:1558] Processing DeleteTablet for tablet f842d8ac6b2142ffa69945c6087be34b with delete_type TABLET_DATA_DELETED (replaced by tablet f1ba93d114e746ef80697aa1401dd108) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.044207 894 tablet_service.cc:1558] Processing DeleteTablet for tablet f842d8ac6b2142ffa69945c6087be34b with delete_type TABLET_DATA_DELETED (replaced by tablet f1ba93d114e746ef80697aa1401dd108) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:11.044270 1348 catalog_manager.cc:6741] ReplaceTablet: tablet f842d8ac6b2142ffa69945c6087be34b deleted and replaced by tablet f1ba93d114e746ef80697aa1401dd108
I20260505 07:54:11.046661 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet f842d8ac6b2142ffa69945c6087be34b with delete_type TABLET_DATA_DELETED (replaced by tablet f1ba93d114e746ef80697aa1401dd108) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.049854 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a403e4708e4725899c1cd5b9fb0e8b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:11.051069 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a403e4708e4725899c1cd5b9fb0e8b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:11.051222 921 raft_consensus.cc:2468] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:11.051397 1176 tablet_bootstrap.cc:492] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:11.052196 751 tablet_service.cc:1558] Processing DeleteTablet for tablet df1b62a30b2842b897a04c793bb5e7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.052868 771 raft_consensus.cc:3060] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.054097 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 02716f62aa354d8f839266661b7f05ea with delete_type TABLET_DATA_DELETED (replaced by tablet 9f541eff399641dd81a0771186805493) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:11.055743 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 297537a92684425d8508f7b3b4c9d379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.055927 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 080b2984fcf2478a8def6540da63554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.055927 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 01431ad112e8445487823efe8feb0dbe) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.056694 912 raft_consensus.cc:1275] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Refusing update from remote peer d63a3e535f8e4dd4a88e447253405b77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260505 07:54:11.057153 1176 tablet_bootstrap.cc:654] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.057737 758 tablet_service.cc:1558] Processing DeleteTablet for tablet f200abef13ac42698a9b96fc8af82ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.057982 758 tablet_service.cc:1558] Processing DeleteTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.058130 751 tablet_service.cc:1558] Processing DeleteTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.058269 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 872362aecf6a44b59e586b1fa7637c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.058266 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 54e4f7f895d943639653c8c32724baf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.060343 757 tablet_service.cc:1558] Processing DeleteTablet for tablet e7887c3462f74740b314e3bec36e280d with delete_type TABLET_DATA_DELETED (replaced by tablet fba1d19f902a4a0a8b581135f6611129) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.060343 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 727cacd242844455915bd9963d2db19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.062299 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 134) took 1630 ms. Trace:
I20260505 07:54:11.062381 2447 rpcz_store.cc:276] 0505 07:54:09.432067 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.432126 (+ 59us) service_pool.cc:224] Handling call
0505 07:54:10.956597 (+1524471us) ts_tablet_manager.cc:1151] Deleting tablet 1602ccbae24544c4a6cf14ea073880b4
0505 07:54:10.956609 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.020342 (+ 63733us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.023172 (+ 2830us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.062282 (+ 39110us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1524248}
I20260505 07:54:11.062677 2447 tablet_replica.cc:333] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:11.063457 2405 leader_election.cc:290] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.063688 991 leader_election.cc:304] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:11.064940 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 1602ccbae24544c4a6cf14ea073880b4 (table default.Table_e9ba8a72d90d4a3da2c06b684fe35033 [id=5c8387e5e7f84731afc62bbea6475a85]) successfully deleted
I20260505 07:54:11.066949 2447 raft_consensus.cc:2243] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:11.071970 1878 consensus_queue.cc:1048] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260505 07:54:11.072337 2302 consensus_queue.cc:1048] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260505 07:54:11.072824 988 leader_election.cc:343] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.072999 2307 raft_consensus.cc:2804] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:11.073226 2307 raft_consensus.cc:493] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:11.073302 2307 raft_consensus.cc:3060] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.074554 2447 raft_consensus.cc:2272] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.075085 2447 ts_tablet_manager.cc:1916] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:11.078420 988 consensus_peers.cc:597] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 297537a92684425d8508f7b3b4c9d379. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.057879 1172 tablet_bootstrap.cc:654] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.080902 781 raft_consensus.cc:3060] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.084950 2444 consensus_queue.cc:1048] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.085822 1878 consensus_queue.cc:1048] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.087675 2307 raft_consensus.cc:515] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.088205 2307 leader_election.cc:290] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.089411 758 tablet_service.cc:1511] Processing CreateTablet for tablet bda7bd7f6fa3493b8f5dc585d83b0de3 (DEFAULT_TABLE table=default.Table_79892c60ca2c461087524d2724724ab6 [id=efcd854d6d3c4205aa9a356f52d4601c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.089453 894 tablet_service.cc:1511] Processing CreateTablet for tablet bda7bd7f6fa3493b8f5dc585d83b0de3 (DEFAULT_TABLE table=default.Table_79892c60ca2c461087524d2724724ab6 [id=efcd854d6d3c4205aa9a356f52d4601c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.089802 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bda7bd7f6fa3493b8f5dc585d83b0de3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.089982 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bda7bd7f6fa3493b8f5dc585d83b0de3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.090581 1023 tablet_service.cc:1511] Processing CreateTablet for tablet bda7bd7f6fa3493b8f5dc585d83b0de3 (DEFAULT_TABLE table=default.Table_79892c60ca2c461087524d2724724ab6 [id=efcd854d6d3c4205aa9a356f52d4601c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.090929 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bda7bd7f6fa3493b8f5dc585d83b0de3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.092360 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a403e4708e4725899c1cd5b9fb0e8b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:11.092612 911 raft_consensus.cc:3060] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.095240 911 raft_consensus.cc:2468] T e1a403e4708e4725899c1cd5b9fb0e8b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:11.098527 1355 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.2:57579 (request call id 154) took 3056 ms (client timeout 1000 ms). Trace:
I20260505 07:54:11.104914 1172 tablet_bootstrap.cc:492] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:11.105063 1172 ts_tablet_manager.cc:1403] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.091s user 0.006s sys 0.000s
I20260505 07:54:11.105863 1172 raft_consensus.cc:359] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.106032 1172 raft_consensus.cc:385] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.106114 1172 raft_consensus.cc:740] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.106297 1172 consensus_queue.cc:260] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.106812 1172 ts_tablet_manager.cc:1434] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:11.107072 1172 tablet_bootstrap.cc:492] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:11.107812 1355 rpcz_store.cc:269] 0505 07:54:08.041745 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:08.106362 (+ 64617us) service_pool.cc:224] Handling call
0505 07:54:11.098483 (+2992121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36898,"reported_tablets":222}
I20260505 07:54:11.118278 991 leader_election.cc:304] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:11.119133 1878 raft_consensus.cc:2804] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:11.119282 1878 raft_consensus.cc:697] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:11.119577 1878 consensus_queue.cc:237] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.122715 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c4fc18f6faa488882d9bffb30793cdd) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.122952 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:11.123492 887 tablet_service.cc:1558] Processing DeleteTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf with delete_type TABLET_DATA_DELETED (replaced by tablet 3a73dbb979344805876efbafbf1b2985) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:11.123605 894 tablet_service.cc:1558] Processing DeleteTablet for tablet a32eeaca77c1475f879465cb1046543b with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:11.124794 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 17327cee69a24e0cb493ff55f948bf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:11.129671 2447 ts_tablet_manager.cc:1929] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.130338 2447 log.cc:1199] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/1838f89dc14a47f99f95f4cc5fdefd70
I20260505 07:54:11.130733 2447 ts_tablet_manager.cc:1950] T 1838f89dc14a47f99f95f4cc5fdefd70 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
W20260505 07:54:11.132028 1354 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.2:57579 (request call id 155) took 1799 ms (client timeout 1000 ms). Trace:
W20260505 07:54:11.133527 1354 rpcz_store.cc:269] 0505 07:54:09.332121 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.446144 (+114023us) service_pool.cc:224] Handling call
0505 07:54:11.131963 (+1685819us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.890678 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.890788 (+ 110us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.890792 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:10.890795 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:10.890869 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.890876 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.890880 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:10.890940 (+ 60us) tablet.cc:701] Row locks acquired
0505 07:54:10.890942 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:10.890969 (+ 27us) write_op.cc:270] Start()
0505 07:54:10.890994 (+ 25us) write_op.cc:276] Timestamp: P: 1777967650890935 usec, L: 0
0505 07:54:10.890997 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.891121 (+ 124us) log.cc:844] Serialized 356 byte log entry
0505 07:54:10.923436 (+ 32315us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.923506 (+ 70us) write_op.cc:301] APPLY: starting
0505 07:54:10.923517 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.923546 (+ 29us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.923573 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.923633 (+ 60us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.923657 (+ 24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:10.923664 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:10.923716 (+ 52us) log.cc:844] Serialized 23 byte log entry
0505 07:54:10.923766 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.923779 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:10.923784 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:11.013648 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.013724 (+ 76us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.013729 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:11.013731 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:11.013797 (+ 66us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.013805 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.013809 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:11.013854 (+ 45us) tablet.cc:701] Row locks acquired
0505 07:54:11.013858 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:11.013884 (+ 26us) write_op.cc:270] Start()
0505 07:54:11.013906 (+ 22us) write_op.cc:276] Timestamp: P: 1777967651013837 usec, L: 0
0505 07:54:11.013909 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.014029 (+ 120us) log.cc:844] Serialized 356 byte log entry
0505 07:54:11.024603 (+ 10574us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.025126 (+ 523us) write_op.cc:301] APPLY: starting
0505 07:54:11.025138 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.025162 (+ 24us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.025180 (+ 18us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.025217 (+ 37us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.025238 (+ 21us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:11.025244 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:11.025293 (+ 49us) log.cc:844] Serialized 23 byte log entry
0505 07:54:11.025327 (+ 34us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.025342 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:11.025347 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:11.025424 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.025428 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":154785,"reported_tablets":253,"child_traces":[["op",{"apply.queue_time_us":30,"apply.run_cpu_time_us":325,"apply.run_wall_time_us":326,"num_ops":1,"prepare.queue_time_us":62,"prepare.run_cpu_time_us":514,"prepare.run_wall_time_us":520,"replication_time_us":32419}],["op",{"apply.queue_time_us":493,"apply.run_cpu_time_us":271,"apply.run_wall_time_us":270,"num_ops":1,"prepare.queue_time_us":39,"prepare.run_cpu_time_us":428,"prepare.run_wall_time_us":429,"replication_time_us":10676}]]}
I20260505 07:54:11.135142 2444 raft_consensus.cc:493] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.135322 2444 raft_consensus.cc:515] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.136061 2444 leader_election.cc:290] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.142561 1172 tablet_bootstrap.cc:654] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.144223 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6a5f823db6143319d743d363a77ea47" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:11.144815 856 leader_election.cc:343] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.148088 1100 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:11.161103 1351 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_a8605cf7bc3b44f39a895bfb7ad9934b" } modify_external_catalogs: true
I20260505 07:54:11.172353 1354 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6ddd9e0382d845d1b89edce006361e1a" } new_table_name: "default.Table_3c38db9697fd4fafa0d5a58a07d4e244" modify_external_catalogs: true
I20260505 07:54:11.173704 1351 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_a8605cf7bc3b44f39a895bfb7ad9934b" } modify_external_catalogs: true
I20260505 07:54:11.175837 1357 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_8f7c0f8991b1415abce7b31407477b52" } new_table_name: "default.Table_4a8d555d38d449c6805a868165a8ab82" modify_external_catalogs: true
I20260505 07:54:11.182622 2276 ts_tablet_manager.cc:1929] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.182888 2276 log.cc:1199] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/9a06f77255544d7dabb71a4f70ada809
I20260505 07:54:11.183621 1355 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7ed74bd3c32f49ef8fbdeb8b065b2913" } modify_external_catalogs: true
I20260505 07:54:11.188380 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet abe5f6f0774545b296bf1b03a08e145d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.188870 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.189347 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.189761 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.190142 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet accb2dedc30e494ebd22c33594cee661 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.190490 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 54e4f7f895d943639653c8c32724baf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.190712 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet ae829daad64545e687a9528e5c38b9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.191637 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet a32eeaca77c1475f879465cb1046543b with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.199116 1355 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7ed74bd3c32f49ef8fbdeb8b065b2913" } modify_external_catalogs: true
I20260505 07:54:11.211295 2268 ts_tablet_manager.cc:1929] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.211495 2268 log.cc:1199] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/872362aecf6a44b59e586b1fa7637c8b
I20260505 07:54:11.211990 2268 ts_tablet_manager.cc:1950] T 872362aecf6a44b59e586b1fa7637c8b P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:11.212366 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 112) took 1650 ms. Trace:
I20260505 07:54:11.212494 2268 rpcz_store.cc:276] 0505 07:54:09.561830 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.566864 (+ 5034us) service_pool.cc:224] Handling call
0505 07:54:10.808088 (+1241224us) ts_tablet_manager.cc:1151] Deleting tablet 872362aecf6a44b59e586b1fa7637c8b
0505 07:54:10.808096 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:10.859973 (+ 51877us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.211209 (+351236us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.212347 (+ 1138us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1241037}
I20260505 07:54:11.213362 2268 tablet_replica.cc:333] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:11.213601 2268 raft_consensus.cc:2243] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:11.214201 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet e278b86e7b5d444ebad8d0f6921d4c62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.214758 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet e6a5f823db6143319d743d363a77ea47 with delete_type TABLET_DATA_DELETED (replaced by tablet 90612f0afb4949f491e625c61e40d247) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.215015 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 25072055f5534078adf5b5025614f951 with delete_type TABLET_DATA_DELETED (replaced by tablet 52e2db0240e04d6d8e948ebaedcc7d24) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.215233 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 72da22d790624b07bfca6f6b6301c925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.215556 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.216311 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet b12d2d2b866348d781209fa787da7d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.217065 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet a9b7c6e9463342a5a73af12da4447f89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.217730 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 85d485ae2d1d4d76a6a5d988cec8291a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.217935 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:11.218962 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 4662d4a3d88d4c2883a35048d6e11eb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:11.219389 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 872362aecf6a44b59e586b1fa7637c8b (table default.Table_4d44519a55e0425d8016d1c0c28ed7f7 [id=4a43764f58e442ad892621ae1f29b214]) successfully deleted
I20260505 07:54:11.227849 2444 raft_consensus.cc:493] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.228041 2444 raft_consensus.cc:515] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.228834 2444 leader_election.cc:290] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:11.219974 1350 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.3:45737 (request call id 158) took 1563 ms (client timeout 1000 ms). Trace:
W20260505 07:54:11.229033 1350 rpcz_store.cc:269] 0505 07:54:09.656547 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.664962 (+ 8415us) service_pool.cc:224] Handling call
0505 07:54:11.219934 (+1554972us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.525026 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.525138 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.525144 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:10.525146 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:10.525317 (+ 171us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.525330 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.525334 (+ 4us) tablet.cc:685] Acquiring locks for 5 operations
0505 07:54:10.525492 (+ 158us) tablet.cc:701] Row locks acquired
0505 07:54:10.525495 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:10.525531 (+ 36us) write_op.cc:270] Start()
0505 07:54:10.525561 (+ 30us) write_op.cc:276] Timestamp: P: 1777967650525479 usec, L: 0
0505 07:54:10.525565 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.525761 (+ 196us) log.cc:844] Serialized 2024 byte log entry
0505 07:54:10.529405 (+ 3644us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.529517 (+ 112us) write_op.cc:301] APPLY: starting
0505 07:54:10.529529 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.529582 (+ 53us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.529608 (+ 26us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.529715 (+ 107us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.529754 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0505 07:54:10.529760 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:10.529848 (+ 88us) log.cc:844] Serialized 47 byte log entry
0505 07:54:10.529909 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.529929 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:10.529934 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:10.530052 (+ 118us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.530057 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":417,"reported_tablets":260,"child_traces":[["op",{"apply.queue_time_us":72,"apply.run_cpu_time_us":492,"apply.run_wall_time_us":492,"num_ops":5,"prepare.queue_time_us":61,"prepare.run_cpu_time_us":853,"prepare.run_wall_time_us":854,"raft.queue_time_us":421,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":253,"replication_time_us":3815}]]}
I20260505 07:54:11.229837 1350 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_58eed87aa11e4e3fa7a7d18a135128a0" } modify_external_catalogs: true
I20260505 07:54:11.230016 1350 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_58eed87aa11e4e3fa7a7d18a135128a0" } modify_external_catalogs: true
I20260505 07:54:11.230584 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:11.231000 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:11.231395 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:11.231631 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:11.233474 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:11.233462 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "139d5a2ab9594f6ea6042c1c4ba8a46a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:11.233986 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:11.234637 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:11.235037 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:11.235244 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:11.235436 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:11.235548 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:11.235721 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:11.235872 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:11.236461 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
W20260505 07:54:11.233767 1352 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.1:36755 (request call id 143) took 2609 ms (client timeout 1000 ms). Trace:
I20260505 07:54:11.237437 1181 tablet_bootstrap.cc:492] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:11.237725 1181 ts_tablet_manager.cc:1403] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.321s user 0.006s sys 0.009s
I20260505 07:54:11.238515 1181 raft_consensus.cc:359] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.238736 1181 raft_consensus.cc:385] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.238858 1181 raft_consensus.cc:740] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.239099 1181 consensus_queue.cc:260] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.239674 1181 ts_tablet_manager.cc:1434] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:11.239984 1181 tablet_bootstrap.cc:492] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:54:11.240178 856 leader_election.cc:343] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.241467 1181 tablet_bootstrap.cc:654] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.247712 2276 ts_tablet_manager.cc:1950] T 9a06f77255544d7dabb71a4f70ada809 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:11.248243 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 71) took 3591 ms. Trace:
I20260505 07:54:11.248415 2276 rpcz_store.cc:276] 0505 07:54:07.657112 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.657159 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:11.029136 (+3371977us) ts_tablet_manager.cc:1151] Deleting tablet 9a06f77255544d7dabb71a4f70ada809
0505 07:54:11.029149 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.034353 (+ 5204us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.182548 (+148195us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.248177 (+ 65629us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3371839}
I20260505 07:54:11.249038 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 9a06f77255544d7dabb71a4f70ada809 (table default.Table_8091d8a01cbf47f3b270874cd5e6140e [id=2a8cda02af654aa5b0c161e04968c735]) successfully deleted
I20260505 07:54:11.249506 2276 tablet_replica.cc:333] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:11.249856 2276 raft_consensus.cc:2243] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:11.250169 2276 raft_consensus.cc:2272] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.250898 2276 ts_tablet_manager.cc:1916] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.256105 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ad8a79245ad47738d079752e51aed2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.256336 749 tablet_service.cc:1558] Processing DeleteTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.256494 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 02716f62aa354d8f839266661b7f05ea with delete_type TABLET_DATA_DELETED (replaced by tablet 9f541eff399641dd81a0771186805493) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.256667 754 tablet_service.cc:1558] Processing DeleteTablet for tablet d622612bdc404727a27b83769bf39738 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.256808 744 tablet_service.cc:1558] Processing DeleteTablet for tablet a32eeaca77c1475f879465cb1046543b with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.257009 754 tablet_service.cc:1558] Processing DeleteTablet for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.257424 749 tablet_service.cc:1558] Processing DeleteTablet for tablet e010f23c4df647ab863e03c3cfed7898 with delete_type TABLET_DATA_DELETED (replaced by tablet 09019fdf60374b11bf3427f21a55b5e0) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.257614 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.257688 2280 raft_consensus.cc:493] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:11.257763 751 tablet_service.cc:1558] Processing DeleteTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:11.257828 2280 raft_consensus.cc:515] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:11.258528 2280 leader_election.cc:290] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:11.236745 1352 rpcz_store.cc:269] 0505 07:54:08.623871 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.021905 (+398034us) service_pool.cc:224] Handling call
0505 07:54:11.233712 (+2211807us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:10.889413 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.889518 (+ 105us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.889523 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:10.889525 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:10.889602 (+ 77us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.889610 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.889614 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:10.889663 (+ 49us) tablet.cc:701] Row locks acquired
0505 07:54:10.889665 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:10.889693 (+ 28us) write_op.cc:270] Start()
0505 07:54:10.889723 (+ 30us) write_op.cc:276] Timestamp: P: 1777967650889659 usec, L: 0
0505 07:54:10.889726 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.889902 (+ 176us) log.cc:844] Serialized 356 byte log entry
0505 07:54:10.922803 (+ 32901us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.922888 (+ 85us) write_op.cc:301] APPLY: starting
0505 07:54:10.922899 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.922930 (+ 31us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.922951 (+ 21us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.922999 (+ 48us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.923024 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:10.923030 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:10.923095 (+ 65us) log.cc:844] Serialized 23 byte log entry
0505 07:54:10.923141 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.923152 (+ 11us) write_op.cc:454] Released schema lock
0505 07:54:10.923157 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:10.935976 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:10.936072 (+ 96us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:10.936077 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:10.936080 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:10.936158 (+ 78us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:10.936165 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:10.936169 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:10.936216 (+ 47us) tablet.cc:701] Row locks acquired
0505 07:54:10.936219 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:10.936248 (+ 29us) write_op.cc:270] Start()
0505 07:54:10.936275 (+ 27us) write_op.cc:276] Timestamp: P: 1777967650936210 usec, L: 0
0505 07:54:10.936280 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:10.936412 (+ 132us) log.cc:844] Serialized 356 byte log entry
0505 07:54:10.969563 (+ 33151us) op_driver.cc:464] REPLICATION: finished
0505 07:54:10.969716 (+ 153us) write_op.cc:301] APPLY: starting
0505 07:54:10.969730 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:10.969767 (+ 37us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:10.969789 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:10.969830 (+ 41us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:10.969859 (+ 29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:10.969866 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:10.969956 (+ 90us) log.cc:844] Serialized 23 byte log entry
0505 07:54:10.970004 (+ 48us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.970021 (+ 17us) write_op.cc:454] Released schema lock
0505 07:54:10.970027 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:10.970086 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:10.970090 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":160803,"reported_tablets":271,"child_traces":[["op",{"apply.queue_time_us":43,"apply.run_cpu_time_us":321,"apply.run_wall_time_us":321,"num_ops":1,"prepare.queue_time_us":62,"prepare.run_cpu_time_us":555,"prepare.run_wall_time_us":556,"replication_time_us":33051}],["op",{"apply.queue_time_us":108,"apply.run_cpu_time_us":363,"apply.run_wall_time_us":363,"num_ops":1,"prepare.queue_time_us":2829,"prepare.run_cpu_time_us":496,"prepare.run_wall_time_us":496,"replication_time_us":33255}]]}
I20260505 07:54:11.264171 1352 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_1f4b1af520d64a108ff4289226a1561d" } new_table_name: "default.Table_bffe918db391471ba1407a66a5b7c240" modify_external_catalogs: true
W20260505 07:54:11.265389 717 leader_election.cc:343] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 8b57c1c618484d5d86a846fb982a2240
I20260505 07:54:11.214120 2268 raft_consensus.cc:2272] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.266534 2268 ts_tablet_manager.cc:1916] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.267243 772 raft_consensus.cc:3060] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.268985 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6a5f823db6143319d743d363a77ea47" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:11.269989 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:11.270305 1358 catalog_manager.cc:5671] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:11.271306 2505 raft_consensus.cc:493] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.271478 2505 raft_consensus.cc:515] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.271867 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:11.272157 2505 leader_election.cc:290] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.272189 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:11.274893 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b4e886bebc4ab6a3c95290d18b4229" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:11.275365 856 leader_election.cc:343] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.276381 2505 raft_consensus.cc:493] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.276510 2505 raft_consensus.cc:515] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:11.276630 854 leader_election.cc:343] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.276777 854 leader_election.cc:304] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.277082 2505 leader_election.cc:290] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.277235 2505 raft_consensus.cc:2749] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.277709 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3e076892234c1494b1a5c13cbfd2e2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:11.278241 854 leader_election.cc:343] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.280334 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:11.280505 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:11.280653 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b0d966bcd7f1419cbd090afc185ae78a (table default.Table_58eed87aa11e4e3fa7a7d18a135128a0 [id=3ae3bc457dcb4db1b5f4360cb9e5c81c]). Expected version 1 got 0
I20260505 07:54:11.280808 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:11.281165 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:11.282634 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:11.283628 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "139d5a2ab9594f6ea6042c1c4ba8a46a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:11.284003 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b4e886bebc4ab6a3c95290d18b4229" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:11.284245 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1a403e4708e4725899c1cd5b9fb0e8b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:11.284543 2414 raft_consensus.cc:493] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:11.284713 2414 raft_consensus.cc:515] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:11.284823 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 42360f8ee4ff47c2af8d287f792207de (table default.Table_b85a2b107c88498e84217252ca6cb947 [id=2c91868ddba84103b9a5c78201842a9e]). Expected version 2 got 1
I20260505 07:54:11.285038 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:11.285400 2414 leader_election.cc:290] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.285521 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 136) took 1837 ms. Trace:
I20260505 07:54:11.285718 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:11.285620 2447 rpcz_store.cc:276] 0505 07:54:09.436007 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.436049 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:11.062637 (+1626588us) ts_tablet_manager.cc:1151] Deleting tablet 1838f89dc14a47f99f95f4cc5fdefd70
0505 07:54:11.062646 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.077997 (+ 15351us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.129190 (+ 51193us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.273303 (+144113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4082,"tablet-delete.queue_time_us":1626437}
I20260505 07:54:11.286017 2447 tablet_replica.cc:333] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:11.285913 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:11.286212 2447 raft_consensus.cc:2243] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:11.286511 2447 raft_consensus.cc:2272] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.286557 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:11.286804 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:11.287051 2447 ts_tablet_manager.cc:1916] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.287163 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:11.287468 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:11.287763 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 21480c42b73e4bf0ab2453b085a867c0 (table default.Table_925028e721e24ba68f31fe1040d4c2fd [id=0faa6676bab04919984b580299143271]). Expected version 1 got 0
I20260505 07:54:11.288020 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:11.288214 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:11.288300 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3e076892234c1494b1a5c13cbfd2e2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:11.288462 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
W20260505 07:54:11.288846 856 leader_election.cc:343] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.288985 856 leader_election.cc:304] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.289182 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:11.289897 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:11.290307 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:11.290345 2505 raft_consensus.cc:2749] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.290510 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:11.290885 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:11.291082 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
W20260505 07:54:11.292344 854 leader_election.cc:343] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.292364 2505 consensus_queue.cc:1048] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.292483 854 leader_election.cc:304] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:11.293087 854 leader_election.cc:343] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.293201 854 leader_election.cc:304] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.293396 2564 raft_consensus.cc:2749] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.293560 2564 raft_consensus.cc:2749] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.294862 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 1838f89dc14a47f99f95f4cc5fdefd70 (table default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958]) successfully deleted
W20260505 07:54:11.305836 717 leader_election.cc:343] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 976359601ad74b149696dc7f555b550c
W20260505 07:54:11.307570 988 leader_election.cc:343] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.308619 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b57c1c618484d5d86a846fb982a2240" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:11.308859 912 raft_consensus.cc:2468] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:11.309214 988 consensus_peers.cc:597] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ba2cc218300472cb5fd6dc8cfb7811c. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:11.315568 718 leader_election.cc:343] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 976359601ad74b149696dc7f555b550c
I20260505 07:54:11.317997 718 leader_election.cc:304] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:11.318403 718 leader_election.cc:304] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:11.318873 2089 raft_consensus.cc:2749] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:11.319562 988 consensus_peers.cc:597] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.321323 2089 raft_consensus.cc:2804] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:11.321416 1172 tablet_bootstrap.cc:492] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:11.321422 2089 raft_consensus.cc:493] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:11.321563 2089 raft_consensus.cc:3060] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:11.321593 1172 ts_tablet_manager.cc:1403] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.215s user 0.008s sys 0.016s
I20260505 07:54:11.322306 1172 raft_consensus.cc:359] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.322533 1172 raft_consensus.cc:385] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.322604 1172 raft_consensus.cc:740] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.322922 1176 tablet_bootstrap.cc:492] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:11.322839 1172 consensus_queue.cc:260] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.323051 1176 ts_tablet_manager.cc:1403] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.272s user 0.028s sys 0.036s
I20260505 07:54:11.323395 1172 ts_tablet_manager.cc:1434] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:11.323750 1176 raft_consensus.cc:359] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.323923 1176 raft_consensus.cc:385] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.324024 1176 raft_consensus.cc:740] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.324206 1176 consensus_queue.cc:260] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.324545 2089 raft_consensus.cc:515] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:11.324783 1172 tablet_bootstrap.cc:492] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:11.325157 2089 leader_election.cc:290] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:11.325850 717 leader_election.cc:343] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 8b57c1c618484d5d86a846fb982a2240
I20260505 07:54:11.328608 775 raft_consensus.cc:3060] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.328908 1176 ts_tablet_manager.cc:1434] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:54:11.329164 1176 tablet_bootstrap.cc:492] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:11.331147 1176 tablet_bootstrap.cc:654] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.333328 2505 consensus_queue.cc:1048] T 4cb7b3fded5d4def8db8d5c1e77ddcda P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.334616 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b57c1c618484d5d86a846fb982a2240" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:11.334784 911 raft_consensus.cc:3060] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Advancing to term 2
W20260505 07:54:11.335398 856 consensus_peers.cc:597] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6da10815e0dc41a1a07b6786882fffd3. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.337895 911 raft_consensus.cc:2468] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 2.
I20260505 07:54:11.338428 718 leader_election.cc:304] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:11.338871 2280 raft_consensus.cc:2804] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:11.338999 2280 raft_consensus.cc:697] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:11.339262 2280 consensus_queue.cc:237] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:11.341472 1176 tablet_bootstrap.cc:492] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:11.341667 1176 ts_tablet_manager.cc:1403] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260505 07:54:11.342526 1176 raft_consensus.cc:359] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.342746 1176 raft_consensus.cc:385] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.342835 1176 raft_consensus.cc:740] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.343076 1176 consensus_queue.cc:260] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.343640 1176 ts_tablet_manager.cc:1434] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260505 07:54:11.343976 1176 tablet_bootstrap.cc:492] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:11.344172 2089 raft_consensus.cc:493] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:11.344359 2089 raft_consensus.cc:515] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:11.345044 2089 leader_election.cc:290] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.345626 1176 tablet_bootstrap.cc:654] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:11.345999 718 leader_election.cc:343] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 43e4c92b7c3140fdb2d6a290aaf0ad55
I20260505 07:54:11.350699 771 raft_consensus.cc:3060] T e79a1da502604d05a62e27f575b8b8f4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.356392 1176 tablet_bootstrap.cc:492] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:11.356516 1176 ts_tablet_manager.cc:1403] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260505 07:54:11.357053 2302 consensus_queue.cc:1048] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.357219 1176 raft_consensus.cc:359] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.357352 1176 raft_consensus.cc:385] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.357430 1176 raft_consensus.cc:740] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.357646 1176 consensus_queue.cc:260] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.358158 1176 ts_tablet_manager.cc:1434] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:11.358426 1176 tablet_bootstrap.cc:492] T e1a403e4708e4725899c1cd5b9fb0e8b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:11.359386 1172 tablet_bootstrap.cc:654] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.360034 1176 tablet_bootstrap.cc:654] T e1a403e4708e4725899c1cd5b9fb0e8b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.327628 2447 ts_tablet_manager.cc:1929] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.362668 2447 log.cc:1199] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/0459f2d45a2d44f1a73e0a0fff53866c
I20260505 07:54:11.363157 2447 ts_tablet_manager.cc:1950] T 0459f2d45a2d44f1a73e0a0fff53866c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:11.366793 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 137) took 1930 ms. Trace:
I20260505 07:54:11.367033 2447 rpcz_store.cc:276] 0505 07:54:09.448274 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.448341 (+ 67us) service_pool.cc:224] Handling call
0505 07:54:11.285954 (+1837613us) ts_tablet_manager.cc:1151] Deleting tablet 0459f2d45a2d44f1a73e0a0fff53866c
0505 07:54:11.285965 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.293377 (+ 7412us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.327500 (+ 34123us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.366760 (+ 39260us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1837349}
I20260505 07:54:11.367611 2447 tablet_replica.cc:333] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:11.367965 2447 raft_consensus.cc:2243] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:11.368144 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 0459f2d45a2d44f1a73e0a0fff53866c (table default.Table_01a5eabcd2ff41fca0d5c94e26775754 [id=0a65eba2d4074216964f45dbb0217d7a]) successfully deleted
I20260505 07:54:11.368229 2447 raft_consensus.cc:2272] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.368863 2447 ts_tablet_manager.cc:1916] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.381798 1172 tablet_bootstrap.cc:492] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:11.382745 1172 ts_tablet_manager.cc:1403] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.058s user 0.001s sys 0.006s
I20260505 07:54:11.389235 1176 tablet_bootstrap.cc:492] T e1a403e4708e4725899c1cd5b9fb0e8b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:11.389417 1176 ts_tablet_manager.cc:1403] T e1a403e4708e4725899c1cd5b9fb0e8b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.007s
I20260505 07:54:11.390676 1176 raft_consensus.cc:359] T e1a403e4708e4725899c1cd5b9fb0e8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.390908 1176 raft_consensus.cc:385] T e1a403e4708e4725899c1cd5b9fb0e8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.411731 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e4c92b7c3140fdb2d6a290aaf0ad55" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:11.414695 2307 consensus_queue.cc:1048] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.418591 717 leader_election.cc:304] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:11.419247 2280 raft_consensus.cc:2749] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:11.420863 1172 raft_consensus.cc:359] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.425233 1172 raft_consensus.cc:385] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.427165 1172 raft_consensus.cc:740] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.429543 758 tablet_service.cc:1511] Processing CreateTablet for tablet e38f248556bd4e49b06da72a1cbd7d8f (DEFAULT_TABLE table=default.Table_7d9b2722b99c42b39ed977c61d37623b [id=12443115e63e4428a9cb798980fcbf5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.429898 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38f248556bd4e49b06da72a1cbd7d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.431756 2405 raft_consensus.cc:493] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.431908 2405 raft_consensus.cc:515] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.432437 2405 leader_election.cc:290] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.431849 1172 consensus_queue.cc:260] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:11.434940 854 consensus_peers.cc:597] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.435609 1034 tablet_service.cc:1511] Processing CreateTablet for tablet e38f248556bd4e49b06da72a1cbd7d8f (DEFAULT_TABLE table=default.Table_7d9b2722b99c42b39ed977c61d37623b [id=12443115e63e4428a9cb798980fcbf5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.436008 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38f248556bd4e49b06da72a1cbd7d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.438416 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54e4f7f895d943639653c8c32724baf2" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:11.438913 988 leader_election.cc:343] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.439203 1176 raft_consensus.cc:740] T e1a403e4708e4725899c1cd5b9fb0e8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.439400 1176 consensus_queue.cc:260] T e1a403e4708e4725899c1cd5b9fb0e8b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:11.439627 988 consensus_peers.cc:597] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1838f89dc14a47f99f95f4cc5fdefd70. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.440241 894 tablet_service.cc:1511] Processing CreateTablet for tablet e38f248556bd4e49b06da72a1cbd7d8f (DEFAULT_TABLE table=default.Table_7d9b2722b99c42b39ed977c61d37623b [id=12443115e63e4428a9cb798980fcbf5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.442349 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e38f248556bd4e49b06da72a1cbd7d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.443728 2444 raft_consensus.cc:493] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.443893 2444 raft_consensus.cc:515] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.444566 2444 leader_election.cc:290] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.445118 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18e909ee93a24ea9842e95357c2ba858" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:11.445538 854 leader_election.cc:343] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.449170 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18e909ee93a24ea9842e95357c2ba858" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:11.449791 856 leader_election.cc:343] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.449913 856 leader_election.cc:304] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.450182 2089 raft_consensus.cc:493] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.450310 2089 raft_consensus.cc:515] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.450387 2444 raft_consensus.cc:2749] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.450939 2089 leader_election.cc:290] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.451258 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a73dbb979344805876efbafbf1b2985" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:11.450821 1172 ts_tablet_manager.cc:1434] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.059s user 0.000s sys 0.004s
I20260505 07:54:11.451540 920 raft_consensus.cc:2393] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:11.452000 1176 ts_tablet_manager.cc:1434] T e1a403e4708e4725899c1cd5b9fb0e8b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.062s user 0.019s sys 0.000s
I20260505 07:54:11.452062 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a73dbb979344805876efbafbf1b2985" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:11.452287 1176 tablet_bootstrap.cc:492] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:11.452848 717 leader_election.cc:304] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:11.453428 2089 raft_consensus.cc:3060] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.453567 758 tablet_service.cc:1511] Processing CreateTablet for tablet d84c071ed258420bbe9d8e603403618c (DEFAULT_TABLE table=default.Table_42065f531a0b4382bfdd52f1b70ebc9d [id=4fd5e25a21c14085b0b7c196336667de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.453891 1176 tablet_bootstrap.cc:654] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.454051 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d84c071ed258420bbe9d8e603403618c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.455276 1172 tablet_bootstrap.cc:492] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:11.456140 2089 raft_consensus.cc:2749] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.456743 2405 consensus_queue.cc:1048] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.458984 1878 consensus_queue.cc:1048] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.461222 1034 tablet_service.cc:1511] Processing CreateTablet for tablet d84c071ed258420bbe9d8e603403618c (DEFAULT_TABLE table=default.Table_42065f531a0b4382bfdd52f1b70ebc9d [id=4fd5e25a21c14085b0b7c196336667de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.461642 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d84c071ed258420bbe9d8e603403618c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.462190 1181 tablet_bootstrap.cc:492] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:11.468026 757 tablet_service.cc:1511] Processing CreateTablet for tablet e81b6dd882844454a9fc9fad1083bb1d (DEFAULT_TABLE table=default.Table_54fb239bdc8941e7bb95f042ae4ee1f7 [id=5306aecfcab94ffd9f68444b8fa17ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.468622 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81b6dd882844454a9fc9fad1083bb1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.468904 1181 ts_tablet_manager.cc:1403] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.229s user 0.004s sys 0.001s
I20260505 07:54:11.470728 2280 raft_consensus.cc:493] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.470891 2280 raft_consensus.cc:515] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.471719 2280 leader_election.cc:290] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.475845 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d442119970a345f28de029eb2139f5c1" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:11.477268 757 tablet_service.cc:1511] Processing CreateTablet for tablet c62136dd7ebc428d9378c3cab8d1ae19 (DEFAULT_TABLE table=default.Table_83f3d8e2b2634dd587c1a08618c17b29 [id=402d96f0bf5d4b97bb4d076af9446b50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.477924 1181 raft_consensus.cc:359] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.478070 1181 raft_consensus.cc:385] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.478377 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c62136dd7ebc428d9378c3cab8d1ae19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.485791 887 tablet_service.cc:1511] Processing CreateTablet for tablet e81b6dd882844454a9fc9fad1083bb1d (DEFAULT_TABLE table=default.Table_54fb239bdc8941e7bb95f042ae4ee1f7 [id=5306aecfcab94ffd9f68444b8fa17ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.486407 899 tablet_service.cc:1511] Processing CreateTablet for tablet c62136dd7ebc428d9378c3cab8d1ae19 (DEFAULT_TABLE table=default.Table_83f3d8e2b2634dd587c1a08618c17b29 [id=402d96f0bf5d4b97bb4d076af9446b50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.488288 2596 tablet.cc:1724] T 21480c42b73e4bf0ab2453b085a867c0 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:11.496408 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d442119970a345f28de029eb2139f5c1" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:11.496680 912 raft_consensus.cc:2393] T d442119970a345f28de029eb2139f5c1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:11.497081 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81b6dd882844454a9fc9fad1083bb1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.497566 718 leader_election.cc:304] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:11.497925 2089 raft_consensus.cc:3060] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.501047 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c62136dd7ebc428d9378c3cab8d1ae19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.501036 2089 raft_consensus.cc:2749] T d442119970a345f28de029eb2139f5c1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.507764 1181 raft_consensus.cc:740] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.508028 1181 consensus_queue.cc:260] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:11.508317 991 leader_election.cc:343] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 54e4f7f895d943639653c8c32724baf2
I20260505 07:54:11.508477 991 leader_election.cc:304] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.508735 2105 raft_consensus.cc:2749] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.463655 2268 ts_tablet_manager.cc:1929] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.484575 2632 tablet.cc:1724] T 21480c42b73e4bf0ab2453b085a867c0 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:11.511415 2268 log.cc:1199] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/6ca2852cee84476dbe5e5072a43087a1
I20260505 07:54:11.487154 1023 tablet_service.cc:1511] Processing CreateTablet for tablet c62136dd7ebc428d9378c3cab8d1ae19 (DEFAULT_TABLE table=default.Table_83f3d8e2b2634dd587c1a08618c17b29 [id=402d96f0bf5d4b97bb4d076af9446b50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.487498 1021 tablet_service.cc:1511] Processing CreateTablet for tablet 3d44e4594e3744319e09b53b1b866636 (DEFAULT_TABLE table=default.Table_3ba75b2103cc434f8eaa19af274c5ff9 [id=54d5399a05584c4da0b0214f0d6aaa60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.511794 2268 ts_tablet_manager.cc:1950] T 6ca2852cee84476dbe5e5072a43087a1 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:11.503298 1025 tablet_service.cc:1511] Processing CreateTablet for tablet e81b6dd882844454a9fc9fad1083bb1d (DEFAULT_TABLE table=default.Table_54fb239bdc8941e7bb95f042ae4ee1f7 [id=5306aecfcab94ffd9f68444b8fa17ef6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.512722 2447 ts_tablet_manager.cc:1929] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.512833 2447 log.cc:1199] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/93d878ae0ddb42319560f74e0a490321
I20260505 07:54:11.513121 2447 ts_tablet_manager.cc:1950] T 93d878ae0ddb42319560f74e0a490321 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:11.503567 2302 raft_consensus.cc:493] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:11.513722 2302 raft_consensus.cc:515] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.514207 2302 leader_election.cc:290] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:11.514892 988 leader_election.cc:343] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: a9494c3b6a99487596a030856b371b68
I20260505 07:54:11.517241 894 tablet_service.cc:1511] Processing CreateTablet for tablet d84c071ed258420bbe9d8e603403618c (DEFAULT_TABLE table=default.Table_42065f531a0b4382bfdd52f1b70ebc9d [id=4fd5e25a21c14085b0b7c196336667de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.520524 1181 ts_tablet_manager.cc:1434] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.049s user 0.001s sys 0.001s
I20260505 07:54:11.520712 2302 raft_consensus.cc:493] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.520901 2302 raft_consensus.cc:515] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.521593 2302 leader_election.cc:290] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.522799 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76939b7978ee4f7ea0ad29312de2a31a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:11.523470 988 leader_election.cc:343] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.524528 758 tablet_service.cc:1511] Processing CreateTablet for tablet 3d44e4594e3744319e09b53b1b866636 (DEFAULT_TABLE table=default.Table_3ba75b2103cc434f8eaa19af274c5ff9 [id=54d5399a05584c4da0b0214f0d6aaa60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.525254 827 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:11.528218 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c62136dd7ebc428d9378c3cab8d1ae19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.528465 915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76939b7978ee4f7ea0ad29312de2a31a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:11.528640 915 raft_consensus.cc:2468] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:11.528959 991 leader_election.cc:343] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: a9494c3b6a99487596a030856b371b68
I20260505 07:54:11.529047 991 leader_election.cc:304] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.529472 2405 raft_consensus.cc:493] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:11.529567 2405 raft_consensus.cc:515] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:11.530038 2405 leader_election.cc:290] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.530436 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e4c92b7c3140fdb2d6a290aaf0ad55" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:11.530571 771 raft_consensus.cc:2468] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:11.530894 988 leader_election.cc:304] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:11.531091 2405 raft_consensus.cc:2804] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:11.531149 2405 raft_consensus.cc:493] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:11.531224 2405 raft_consensus.cc:3060] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:11.531793 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d44e4594e3744319e09b53b1b866636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.533655 2405 raft_consensus.cc:515] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:11.534096 2405 leader_election.cc:290] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.534596 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43e4c92b7c3140fdb2d6a290aaf0ad55" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:11.534736 771 raft_consensus.cc:3060] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:11.537110 771 raft_consensus.cc:2468] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 2.
I20260505 07:54:11.537506 988 leader_election.cc:304] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:11.537796 2405 raft_consensus.cc:2804] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:11.538015 2405 raft_consensus.cc:697] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:11.538291 2405 consensus_queue.cc:237] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:11.539177 2302 raft_consensus.cc:2749] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:11.539614 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e81b6dd882844454a9fc9fad1083bb1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.540021 1181 tablet_bootstrap.cc:492] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:11.540540 991 leader_election.cc:304] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.541404 1878 raft_consensus.cc:2804] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:11.541471 1878 raft_consensus.cc:493] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:11.541522 1878 raft_consensus.cc:3060] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.541611 1181 tablet_bootstrap.cc:654] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.542253 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 113) took 1975 ms. Trace:
I20260505 07:54:11.542325 2268 rpcz_store.cc:276] 0505 07:54:09.567221 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.573062 (+ 5841us) service_pool.cc:224] Handling call
0505 07:54:11.213311 (+1640249us) ts_tablet_manager.cc:1151] Deleting tablet 6ca2852cee84476dbe5e5072a43087a1
0505 07:54:11.213320 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.409419 (+196099us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.463609 (+ 54190us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.542235 (+ 78626us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1640018}
I20260505 07:54:11.543076 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 114) took 1969 ms. Trace:
I20260505 07:54:11.543149 2268 rpcz_store.cc:276] 0505 07:54:09.573458 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.573502 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:11.542559 (+1969057us) ts_tablet_manager.cc:1151] Deleting tablet df1b62a30b2842b897a04c793bb5e7ce
0505 07:54:11.542568 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.543057 (+ 489us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1968831}
I20260505 07:54:11.543413 2268 tablet_replica.cc:333] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:11.543622 2268 raft_consensus.cc:2243] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:11.543841 1878 raft_consensus.cc:515] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.543941 2268 raft_consensus.cc:2272] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.544368 1878 leader_election.cc:290] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.544526 2268 ts_tablet_manager.cc:1916] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:11.544728 2628 log.cc:927] Time spent T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: Append to log took a long time: real 0.082s user 0.000s sys 0.000s
W20260505 07:54:11.545490 991 leader_election.cc:343] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 43e4c92b7c3140fdb2d6a290aaf0ad55
W20260505 07:54:11.545862 991 leader_election.cc:343] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 43e4c92b7c3140fdb2d6a290aaf0ad55
I20260505 07:54:11.546527 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 6ca2852cee84476dbe5e5072a43087a1 (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]) successfully deleted
W20260505 07:54:11.546830 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet df1b62a30b2842b897a04c793bb5e7ce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df1b62a30b2842b897a04c793bb5e7ce already in progress: creating tablet
I20260505 07:54:11.548080 1176 tablet_bootstrap.cc:492] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:11.548291 1176 ts_tablet_manager.cc:1403] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.096s user 0.007s sys 0.019s
I20260505 07:54:11.548672 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76939b7978ee4f7ea0ad29312de2a31a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:11.548851 916 raft_consensus.cc:3060] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.548846 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76939b7978ee4f7ea0ad29312de2a31a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:11.548946 1176 raft_consensus.cc:359] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.549079 1176 raft_consensus.cc:385] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.549149 1176 raft_consensus.cc:740] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.549258 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d84c071ed258420bbe9d8e603403618c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:11.549264 988 leader_election.cc:343] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:11.549765 988 consensus_peers.cc:597] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1602ccbae24544c4a6cf14ea073880b4. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.549731 1176 consensus_queue.cc:260] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.550212 1176 ts_tablet_manager.cc:1434] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:11.551177 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d44e4594e3744319e09b53b1b866636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.552075 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 138) took 2100 ms. Trace:
I20260505 07:54:11.552193 2447 rpcz_store.cc:276] 0505 07:54:09.451742 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.451804 (+ 62us) service_pool.cc:224] Handling call
0505 07:54:11.367514 (+1915710us) ts_tablet_manager.cc:1151] Deleting tablet 93d878ae0ddb42319560f74e0a490321
0505 07:54:11.367527 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.418773 (+ 51246us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.512655 (+ 93882us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.552060 (+ 39405us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1915516}
I20260505 07:54:11.552659 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 139) took 2100 ms. Trace:
I20260505 07:54:11.552654 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 93d878ae0ddb42319560f74e0a490321 (table default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]) successfully deleted
I20260505 07:54:11.552789 2447 rpcz_store.cc:276] 0505 07:54:09.452162 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.452209 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:11.552589 (+2100380us) ts_tablet_manager.cc:1151] Deleting tablet 85d485ae2d1d4d76a6a5d988cec8291a
0505 07:54:11.552597 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.552645 (+ 48us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2100241}
I20260505 07:54:11.553202 2447 tablet_replica.cc:333] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:11.553412 2447 raft_consensus.cc:2243] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:11.553406 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 85d485ae2d1d4d76a6a5d988cec8291a because the tablet was not found. No further retry: Not found: Tablet not found: 85d485ae2d1d4d76a6a5d988cec8291a
I20260505 07:54:11.553838 1176 tablet_bootstrap.cc:492] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:11.553606 2447 raft_consensus.cc:2272] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.554560 2447 ts_tablet_manager.cc:1916] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.555330 916 raft_consensus.cc:2468] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:11.555454 2564 raft_consensus.cc:493] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.555568 2564 raft_consensus.cc:515] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:11.556212 2564 leader_election.cc:290] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.559916 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe19306f7df49fdb771d21f6560e697" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:11.561852 856 leader_election.cc:343] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.563567 1181 tablet_bootstrap.cc:492] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:11.563802 1181 ts_tablet_manager.cc:1403] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.002s
I20260505 07:54:11.564711 1181 raft_consensus.cc:359] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.564962 1181 raft_consensus.cc:385] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.565073 1878 raft_consensus.cc:493] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.565253 1878 raft_consensus.cc:515] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.565769 1878 leader_election.cc:290] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.565100 1181 raft_consensus.cc:740] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.567718 991 leader_election.cc:304] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.567806 1181 consensus_queue.cc:260] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.568167 2105 raft_consensus.cc:2804] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:11.568280 2105 raft_consensus.cc:697] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:11.568454 1181 ts_tablet_manager.cc:1434] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260505 07:54:11.568497 2105 consensus_queue.cc:237] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.568750 1181 tablet_bootstrap.cc:492] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:11.570847 1181 tablet_bootstrap.cc:654] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.585906 2276 ts_tablet_manager.cc:1929] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.586241 2276 log.cc:1199] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/a0f2f46d0e7840458cb992ce2a2dbea2
I20260505 07:54:11.595710 1172 tablet_bootstrap.cc:654] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.597360 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e010f23c4df647ab863e03c3cfed7898" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:11.597523 911 raft_consensus.cc:2468] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:11.598563 2268 ts_tablet_manager.cc:1929] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.601673 2268 log.cc:1199] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/297537a92684425d8508f7b3b4c9d379
I20260505 07:54:11.602214 2268 ts_tablet_manager.cc:1950] T 297537a92684425d8508f7b3b4c9d379 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:11.555089 1176 tablet_bootstrap.cc:654] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.556703 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe19306f7df49fdb771d21f6560e697" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:11.603724 854 leader_election.cc:343] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.603839 854 leader_election.cc:304] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.604058 2564 raft_consensus.cc:2749] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.566473 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e010f23c4df647ab863e03c3cfed7898" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:11.594621 781 raft_consensus.cc:3060] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.600344 991 leader_election.cc:304] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:11.608238 2438 raft_consensus.cc:2804] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:11.608403 2438 raft_consensus.cc:493] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:11.608491 2438 raft_consensus.cc:3060] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.610999 2438 raft_consensus.cc:515] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.612339 2438 leader_election.cc:290] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:11.614557 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e010f23c4df647ab863e03c3cfed7898" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:11.616245 988 leader_election.cc:343] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.602602 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 115) took 2023 ms. Trace:
I20260505 07:54:11.617884 2268 rpcz_store.cc:276] 0505 07:54:09.579496 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.579545 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:11.543373 (+1963828us) ts_tablet_manager.cc:1151] Deleting tablet 297537a92684425d8508f7b3b4c9d379
0505 07:54:11.543383 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.547556 (+ 4173us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.598506 (+ 50950us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.602583 (+ 4077us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1963538}
I20260505 07:54:11.618505 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 116) took 2038 ms. Trace:
I20260505 07:54:11.618605 2268 rpcz_store.cc:276] 0505 07:54:09.580013 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.580058 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:11.618263 (+2038205us) ts_tablet_manager.cc:1151] Deleting tablet 080b2984fcf2478a8def6540da63554d
0505 07:54:11.618279 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.618481 (+ 202us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2037990}
I20260505 07:54:11.619036 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 117) took 2038 ms. Trace:
I20260505 07:54:11.619125 2268 rpcz_store.cc:276] 0505 07:54:09.580719 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.580766 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:11.618881 (+2038115us) ts_tablet_manager.cc:1151] Deleting tablet f200abef13ac42698a9b96fc8af82ff0
0505 07:54:11.618892 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.619020 (+ 128us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2037878}
I20260505 07:54:11.619500 2268 tablet_replica.cc:333] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:11.619827 2268 raft_consensus.cc:2243] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:11.620040 2268 raft_consensus.cc:2272] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.620493 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 297537a92684425d8508f7b3b4c9d379 (table default.Table_44ead87f92584e6e98e2c5f69056591a [id=e78409bbbd334198b96cc757bc1feee8]) successfully deleted
W20260505 07:54:11.620863 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 080b2984fcf2478a8def6540da63554d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 080b2984fcf2478a8def6540da63554d already in progress: creating tablet
W20260505 07:54:11.621120 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f200abef13ac42698a9b96fc8af82ff0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f200abef13ac42698a9b96fc8af82ff0 already in progress: creating tablet
W20260505 07:54:11.625759 988 leader_election.cc:343] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.636257 1176 tablet_bootstrap.cc:492] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:11.636392 1176 ts_tablet_manager.cc:1403] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.083s user 0.005s sys 0.014s
I20260505 07:54:11.637192 1176 raft_consensus.cc:359] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.637346 1176 raft_consensus.cc:385] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.637390 1176 raft_consensus.cc:740] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.637552 1176 consensus_queue.cc:260] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.638020 1176 ts_tablet_manager.cc:1434] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:11.638099 757 tablet_service.cc:1511] Processing CreateTablet for tablet 830781656e2541108a78a4807aef7bea (DEFAULT_TABLE table=default.Table_91e9ab2f967244e6a89f8c76b0f58e93 [id=e77e766d5ccc47c0b0cac6f76dd090c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.639258 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 830781656e2541108a78a4807aef7bea (DEFAULT_TABLE table=default.Table_91e9ab2f967244e6a89f8c76b0f58e93 [id=e77e766d5ccc47c0b0cac6f76dd090c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.639499 899 tablet_service.cc:1511] Processing CreateTablet for tablet 830781656e2541108a78a4807aef7bea (DEFAULT_TABLE table=default.Table_91e9ab2f967244e6a89f8c76b0f58e93 [id=e77e766d5ccc47c0b0cac6f76dd090c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.639751 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 830781656e2541108a78a4807aef7bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.640046 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 830781656e2541108a78a4807aef7bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.646795 894 tablet_service.cc:1511] Processing CreateTablet for tablet 3d44e4594e3744319e09b53b1b866636 (DEFAULT_TABLE table=default.Table_3ba75b2103cc434f8eaa19af274c5ff9 [id=54d5399a05584c4da0b0214f0d6aaa60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.646993 751 tablet_service.cc:1511] Processing CreateTablet for tablet d5d3b44cac3f47ff82d815a8f551eb2a (DEFAULT_TABLE table=default.Table_2a1375626b524e7fa087a6db2c2783f8 [id=a1afd760fa4b45e2af21aed843a08caa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.647351 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d44e4594e3744319e09b53b1b866636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.651765 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e010f23c4df647ab863e03c3cfed7898" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:11.652000 921 raft_consensus.cc:3060] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.654232 1176 tablet_bootstrap.cc:492] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:11.654913 921 raft_consensus.cc:2468] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:11.656006 1176 tablet_bootstrap.cc:654] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.656513 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 830781656e2541108a78a4807aef7bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.656667 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5d3b44cac3f47ff82d815a8f551eb2a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:11.663918 988 consensus_peers.cc:597] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ad8a79245ad47738d079752e51aed2b. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.666354 1176 tablet_bootstrap.cc:492] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:11.666620 1176 ts_tablet_manager.cc:1403] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260505 07:54:11.669281 1176 raft_consensus.cc:359] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.669711 1176 raft_consensus.cc:385] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.669904 1176 raft_consensus.cc:740] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.670434 1176 consensus_queue.cc:260] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.671635 1176 ts_tablet_manager.cc:1434] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260505 07:54:11.672128 1176 tablet_bootstrap.cc:492] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:11.674844 1176 tablet_bootstrap.cc:654] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.656266 2276 ts_tablet_manager.cc:1950] T a0f2f46d0e7840458cb992ce2a2dbea2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:11.680713 1172 tablet_bootstrap.cc:492] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:11.688539 1172 ts_tablet_manager.cc:1403] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.234s user 0.003s sys 0.003s
I20260505 07:54:11.688925 1025 tablet_service.cc:1511] Processing CreateTablet for tablet d5d3b44cac3f47ff82d815a8f551eb2a (DEFAULT_TABLE table=default.Table_2a1375626b524e7fa087a6db2c2783f8 [id=a1afd760fa4b45e2af21aed843a08caa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.689318 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5d3b44cac3f47ff82d815a8f551eb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.689314 1172 raft_consensus.cc:359] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.689476 1172 raft_consensus.cc:385] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.689539 1172 raft_consensus.cc:740] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.689735 1172 consensus_queue.cc:260] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.689930 899 tablet_service.cc:1511] Processing CreateTablet for tablet d5d3b44cac3f47ff82d815a8f551eb2a (DEFAULT_TABLE table=default.Table_2a1375626b524e7fa087a6db2c2783f8 [id=a1afd760fa4b45e2af21aed843a08caa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.690354 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5d3b44cac3f47ff82d815a8f551eb2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.690354 1172 ts_tablet_manager.cc:1434] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:11.690637 1172 tablet_bootstrap.cc:492] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:11.687562 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:11.694890 1172 tablet_bootstrap.cc:654] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.696151 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 72) took 4031 ms. Trace:
I20260505 07:54:11.696350 2276 rpcz_store.cc:276] 0505 07:54:07.658604 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.658712 (+ 108us) service_pool.cc:224] Handling call
0505 07:54:11.249397 (+3590685us) ts_tablet_manager.cc:1151] Deleting tablet a0f2f46d0e7840458cb992ce2a2dbea2
0505 07:54:11.249410 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.254724 (+ 5314us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.585771 (+331047us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.689645 (+103874us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3590342}
I20260505 07:54:11.697043 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet a0f2f46d0e7840458cb992ce2a2dbea2 (table default.Table_f8f7a899e8c94e75958bd11fed643bc6 [id=a3cea9a70f4147b7bf5b630980c94c27]) successfully deleted
I20260505 07:54:11.697461 2276 tablet_replica.cc:333] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:11.697925 2276 raft_consensus.cc:2243] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:11.698158 2276 raft_consensus.cc:2272] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.698868 2276 ts_tablet_manager.cc:1916] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.700446 757 tablet_service.cc:1511] Processing CreateTablet for tablet b9a92043febc4c6481523bea8f75ab5e (DEFAULT_TABLE table=default.Table_80c5d2cc30df4219ac6df63476fe3e14 [id=c9eef640ca7d465380672ff81a3f9db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.701364 2302 consensus_queue.cc:1048] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.704918 885 tablet_service.cc:1511] Processing CreateTablet for tablet b9a92043febc4c6481523bea8f75ab5e (DEFAULT_TABLE table=default.Table_80c5d2cc30df4219ac6df63476fe3e14 [id=c9eef640ca7d465380672ff81a3f9db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.705243 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9a92043febc4c6481523bea8f75ab5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.709280 751 tablet_service.cc:1511] Processing CreateTablet for tablet 03260a8507804e548486c81225de3ad9 (DEFAULT_TABLE table=default.Table_d798bdca20804d6eb6aa52b61c2b69e3 [id=3c21c289d8f94c70b4ba7ef64dabc072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.710309 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 03260a8507804e548486c81225de3ad9 (DEFAULT_TABLE table=default.Table_d798bdca20804d6eb6aa52b61c2b69e3 [id=3c21c289d8f94c70b4ba7ef64dabc072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.710642 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03260a8507804e548486c81225de3ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.717947 2268 ts_tablet_manager.cc:1916] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.719774 1025 tablet_service.cc:1511] Processing CreateTablet for tablet b9a92043febc4c6481523bea8f75ab5e (DEFAULT_TABLE table=default.Table_80c5d2cc30df4219ac6df63476fe3e14 [id=c9eef640ca7d465380672ff81a3f9db6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.721122 991 leader_election.cc:304] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.721164 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9a92043febc4c6481523bea8f75ab5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.722342 2438 raft_consensus.cc:2804] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:11.722620 2105 consensus_queue.cc:1048] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.722991 2105 consensus_queue.cc:1048] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.730144 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03260a8507804e548486c81225de3ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.730305 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9a92043febc4c6481523bea8f75ab5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.733563 2564 raft_consensus.cc:493] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.733711 2564 raft_consensus.cc:515] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.734244 2564 leader_election.cc:290] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.734509 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0d43d85fcd4d05abf10d415e8dafbe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:11.734818 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0d43d85fcd4d05abf10d415e8dafbe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:11.735217 856 leader_election.cc:343] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:11.735387 854 leader_election.cc:343] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.735478 854 leader_election.cc:304] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.735929 2564 raft_consensus.cc:2749] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.740500 2447 ts_tablet_manager.cc:1929] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.740600 2447 log.cc:1199] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/6da10815e0dc41a1a07b6786882fffd3
I20260505 07:54:11.740964 2447 ts_tablet_manager.cc:1950] T 6da10815e0dc41a1a07b6786882fffd3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:11.741313 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 140) took 2288 ms. Trace:
I20260505 07:54:11.741397 2447 rpcz_store.cc:276] 0505 07:54:09.452471 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.452515 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:11.553164 (+2100649us) ts_tablet_manager.cc:1151] Deleting tablet 6da10815e0dc41a1a07b6786882fffd3
0505 07:54:11.553172 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.730130 (+176958us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.740458 (+ 10328us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.741299 (+ 841us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2100453}
I20260505 07:54:11.741806 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 6da10815e0dc41a1a07b6786882fffd3 (table default.Table_344c494090d44fa8b4102b525e7ca099 [id=d8050f3674fe49f7a8af288e6ea57a69]) successfully deleted
I20260505 07:54:11.742910 1181 tablet_bootstrap.cc:492] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:11.743144 892 tablet_service.cc:1511] Processing CreateTablet for tablet 03260a8507804e548486c81225de3ad9 (DEFAULT_TABLE table=default.Table_d798bdca20804d6eb6aa52b61c2b69e3 [id=3c21c289d8f94c70b4ba7ef64dabc072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.743494 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03260a8507804e548486c81225de3ad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.743659 1181 ts_tablet_manager.cc:1403] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.175s user 0.022s sys 0.018s
I20260505 07:54:11.746829 751 tablet_service.cc:1511] Processing CreateTablet for tablet f1ba93d114e746ef80697aa1401dd108 (DEFAULT_TABLE table=default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.747455 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1ba93d114e746ef80697aa1401dd108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.749724 899 tablet_service.cc:1511] Processing CreateTablet for tablet f1ba93d114e746ef80697aa1401dd108 (DEFAULT_TABLE table=default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.745486 1181 raft_consensus.cc:359] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.760798 1181 raft_consensus.cc:385] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.761389 1181 raft_consensus.cc:740] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.762290 2652 tablet.cc:1724] T 21480c42b73e4bf0ab2453b085a867c0 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:11.762863 2438 raft_consensus.cc:697] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:11.766597 1181 consensus_queue.cc:260] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:11.766851 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 44470.1us
I20260505 07:54:11.766734 2438 consensus_queue.cc:237] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.768586 1181 ts_tablet_manager.cc:1434] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20260505 07:54:11.770519 1034 tablet_service.cc:1511] Processing CreateTablet for tablet f1ba93d114e746ef80697aa1401dd108 (DEFAULT_TABLE table=default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:11.770958 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1ba93d114e746ef80697aa1401dd108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.771602 1181 tablet_bootstrap.cc:492] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:11.772285 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1ba93d114e746ef80697aa1401dd108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:11.777848 2447 tablet_replica.cc:333] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:11.778069 2447 raft_consensus.cc:2243] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:11.778360 2447 raft_consensus.cc:2272] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.783568 2447 ts_tablet_manager.cc:1916] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.784698 2268 ts_tablet_manager.cc:1929] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:11.784873 2268 log.cc:1199] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/472441bd29704be59aabb0f7a4964bb0
I20260505 07:54:11.785285 1181 tablet_bootstrap.cc:654] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.786688 2564 raft_consensus.cc:493] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.786804 2564 raft_consensus.cc:515] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.787289 2564 leader_election.cc:290] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.788015 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec145bdc0dab4642ba233bd0da2c436f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:11.785295 2268 ts_tablet_manager.cc:1950] T 472441bd29704be59aabb0f7a4964bb0 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
W20260505 07:54:11.788486 854 leader_election.cc:343] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.789638 1172 tablet_bootstrap.cc:492] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:11.789760 1172 ts_tablet_manager.cc:1403] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.099s user 0.009s sys 0.000s
I20260505 07:54:11.790561 1172 raft_consensus.cc:359] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.790746 1172 raft_consensus.cc:385] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.790808 1172 raft_consensus.cc:740] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.791010 1172 consensus_queue.cc:260] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.791534 1172 ts_tablet_manager.cc:1434] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:11.793651 1172 tablet_bootstrap.cc:492] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:11.795066 1172 tablet_bootstrap.cc:654] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.795854 1176 tablet_bootstrap.cc:492] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:11.796176 1176 ts_tablet_manager.cc:1403] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.124s user 0.010s sys 0.000s
I20260505 07:54:11.797703 1176 raft_consensus.cc:359] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.798095 1176 raft_consensus.cc:385] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.798296 1176 raft_consensus.cc:740] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.803352 2632 tablet.cc:1724] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:11.803893 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec145bdc0dab4642ba233bd0da2c436f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:11.804804 856 leader_election.cc:343] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.804926 856 leader_election.cc:304] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.805296 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 118) took 2224 ms. Trace:
I20260505 07:54:11.805402 2268 rpcz_store.cc:276] 0505 07:54:09.581170 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.581213 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:11.619367 (+2038154us) ts_tablet_manager.cc:1151] Deleting tablet 472441bd29704be59aabb0f7a4964bb0
0505 07:54:11.619452 (+ 85us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.748863 (+129411us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.784647 (+ 35784us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.805274 (+ 20627us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15893,"tablet-delete.queue_time_us":2037848}
I20260505 07:54:11.805857 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 119) took 2220 ms. Trace:
I20260505 07:54:11.805927 2268 rpcz_store.cc:276] 0505 07:54:09.585378 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.585424 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:11.805711 (+2220287us) ts_tablet_manager.cc:1151] Deleting tablet e7887c3462f74740b314e3bec36e280d
0505 07:54:11.805724 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.805840 (+ 116us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2220033}
I20260505 07:54:11.806198 2268 tablet_replica.cc:333] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:11.806435 2268 raft_consensus.cc:2243] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:11.806701 2268 pending_rounds.cc:70] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:54:11.806748 2268 pending_rounds.cc:77] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555497389105152 op_type: NO_OP noop_request { }
I20260505 07:54:11.806864 2268 raft_consensus.cc:2889] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:11.806957 2268 raft_consensus.cc:2272] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.807108 2564 raft_consensus.cc:2749] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.807606 2268 ts_tablet_manager.cc:1916] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:11.809450 854 consensus_peers.cc:597] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6da10815e0dc41a1a07b6786882fffd3. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.809516 2596 tablet.cc:1724] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:11.815349 2276 ts_tablet_manager.cc:1929] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:11.815488 2276 log.cc:1199] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/0db162dfad2448e8ac428b3d3fbe17f7
I20260505 07:54:11.815825 2276 ts_tablet_manager.cc:1950] T 0db162dfad2448e8ac428b3d3fbe17f7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:11.816586 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 74) took 4155 ms. Trace:
I20260505 07:54:11.803053 1176 consensus_queue.cc:260] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.818429 1176 ts_tablet_manager.cc:1434] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.022s user 0.003s sys 0.000s
I20260505 07:54:11.819298 1176 tablet_bootstrap.cc:492] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:11.824954 2280 raft_consensus.cc:493] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.825067 2280 raft_consensus.cc:515] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.825635 2280 leader_election.cc:290] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.826730 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b4e886bebc4ab6a3c95290d18b4229" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:11.826905 1051 raft_consensus.cc:2468] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:11.827302 1181 tablet_bootstrap.cc:492] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:11.827487 1181 ts_tablet_manager.cc:1403] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.056s user 0.003s sys 0.006s
I20260505 07:54:11.827497 717 leader_election.cc:304] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:11.827761 2280 raft_consensus.cc:2804] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:11.827834 2280 raft_consensus.cc:493] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:11.827911 2280 raft_consensus.cc:3060] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.828469 1181 raft_consensus.cc:359] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.828688 1181 raft_consensus.cc:385] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.828784 1181 raft_consensus.cc:740] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.829088 1181 consensus_queue.cc:260] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.830180 1181 ts_tablet_manager.cc:1434] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:11.830896 2280 raft_consensus.cc:515] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.832294 2280 leader_election.cc:290] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.832863 2276 rpcz_store.cc:276] 0505 07:54:07.660939 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.661089 (+ 150us) service_pool.cc:224] Handling call
0505 07:54:11.697408 (+4036319us) ts_tablet_manager.cc:1151] Deleting tablet 0db162dfad2448e8ac428b3d3fbe17f7
0505 07:54:11.697421 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.770072 (+ 72651us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.815304 (+ 45232us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.816557 (+ 1253us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4036171}
I20260505 07:54:11.834980 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 472441bd29704be59aabb0f7a4964bb0 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]) successfully deleted
W20260505 07:54:11.835289 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e7887c3462f74740b314e3bec36e280d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7887c3462f74740b314e3bec36e280d already in progress: creating tablet
I20260505 07:54:11.836359 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 0db162dfad2448e8ac428b3d3fbe17f7 (table default.Table_73ba90aede024514abb230661a44a4ae [id=03a241143fae44a19531301524ef4adb]) successfully deleted
I20260505 07:54:11.821601 2564 raft_consensus.cc:493] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.836589 2564 raft_consensus.cc:515] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.837081 2276 tablet_replica.cc:333] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:11.837142 2564 leader_election.cc:290] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.837272 2276 raft_consensus.cc:2243] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:11.837419 2276 raft_consensus.cc:2272] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.831815 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b4e886bebc4ab6a3c95290d18b4229" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:11.820075 967 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:11.837996 2276 ts_tablet_manager.cc:1916] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.838037 912 raft_consensus.cc:3060] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.840694 912 raft_consensus.cc:2468] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:11.841465 718 leader_election.cc:304] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:11.841920 1181 tablet_bootstrap.cc:492] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:11.842037 2089 raft_consensus.cc:2804] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:11.842145 2089 raft_consensus.cc:697] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:11.842351 2089 consensus_queue.cc:237] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.843742 1181 tablet_bootstrap.cc:654] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.843873 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f200abef13ac42698a9b96fc8af82ff0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:11.844324 856 leader_election.cc:343] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.832695 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b4e886bebc4ab6a3c95290d18b4229" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:11.844769 1051 raft_consensus.cc:3060] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.845556 1172 tablet_bootstrap.cc:492] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:11.845669 1172 ts_tablet_manager.cc:1403] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.052s user 0.001s sys 0.007s
I20260505 07:54:11.846261 1172 raft_consensus.cc:359] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.846397 1172 raft_consensus.cc:385] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.846442 1172 raft_consensus.cc:740] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.846607 1172 consensus_queue.cc:260] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.847092 1172 ts_tablet_manager.cc:1434] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:11.847340 1172 tablet_bootstrap.cc:492] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:11.848134 988 consensus_peers.cc:597] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d622612bdc404727a27b83769bf39738. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.849565 1172 tablet_bootstrap.cc:654] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.849992 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f200abef13ac42698a9b96fc8af82ff0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:11.850394 854 leader_election.cc:343] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.850504 854 leader_election.cc:304] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.850908 1181 tablet_bootstrap.cc:492] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:11.851068 1181 ts_tablet_manager.cc:1403] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.002s
I20260505 07:54:11.851460 2564 raft_consensus.cc:2749] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.851480 2447 ts_tablet_manager.cc:1929] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.851825 1181 raft_consensus.cc:359] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.852036 1181 raft_consensus.cc:385] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.852136 1181 raft_consensus.cc:740] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.851997 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17b4e886bebc4ab6a3c95290d18b4229" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:11.852385 1181 consensus_queue.cc:260] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.852375 921 raft_consensus.cc:2376] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1. Re-sending same reply.
I20260505 07:54:11.853188 1181 ts_tablet_manager.cc:1434] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:11.847551 1051 raft_consensus.cc:2468] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:11.853470 1181 tablet_bootstrap.cc:492] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:11.855115 1181 tablet_bootstrap.cc:654] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.855738 2564 raft_consensus.cc:493] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.855850 2564 raft_consensus.cc:515] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.856357 2564 leader_election.cc:290] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.857228 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f842d8ac6b2142ffa69945c6087be34b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:11.857704 856 leader_election.cc:343] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.857952 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f842d8ac6b2142ffa69945c6087be34b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:11.858311 854 leader_election.cc:343] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.858428 854 leader_election.cc:304] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.858939 2564 raft_consensus.cc:2749] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.862704 1181 tablet_bootstrap.cc:492] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:11.863170 1181 ts_tablet_manager.cc:1403] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.001s
I20260505 07:54:11.864033 1181 raft_consensus.cc:359] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.864867 1176 tablet_bootstrap.cc:654] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.865195 1181 raft_consensus.cc:385] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.865387 1181 raft_consensus.cc:740] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.865760 1181 consensus_queue.cc:260] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.866684 1181 ts_tablet_manager.cc:1434] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:11.867337 1181 tablet_bootstrap.cc:492] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:11.869688 1181 tablet_bootstrap.cc:654] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.871340 2652 tablet.cc:1724] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:11.871888 2280 raft_consensus.cc:493] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.872010 2280 raft_consensus.cc:515] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.873109 2280 leader_election.cc:290] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.873538 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54e4f7f895d943639653c8c32724baf2" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:11.873705 1051 raft_consensus.cc:2468] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:11.874096 717 leader_election.cc:304] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:11.874332 2280 raft_consensus.cc:2804] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:11.874397 2280 raft_consensus.cc:493] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:11.874459 2280 raft_consensus.cc:3060] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.851714 2447 log.cc:1199] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/12501fdf2cc5491cb1a06db490e70810
I20260505 07:54:11.879722 2447 ts_tablet_manager.cc:1950] T 12501fdf2cc5491cb1a06db490e70810 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
W20260505 07:54:11.882788 718 leader_election.cc:343] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 54e4f7f895d943639653c8c32724baf2
I20260505 07:54:11.883002 1181 tablet_bootstrap.cc:492] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:11.883554 1181 ts_tablet_manager.cc:1403] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.002s
I20260505 07:54:11.883204 2402 consensus_queue.cc:1048] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:11.884191 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 141) took 2431 ms. Trace:
I20260505 07:54:11.884717 2564 raft_consensus.cc:493] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.884819 2564 raft_consensus.cc:515] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.885619 2564 leader_election.cc:290] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.886451 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b46ff0101a4e529077b9e30492ace7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:11.886504 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b46ff0101a4e529077b9e30492ace7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:11.886981 856 leader_election.cc:343] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:11.887090 854 leader_election.cc:343] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.887197 854 leader_election.cc:304] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:11.885037 2447 rpcz_store.cc:276] 0505 07:54:09.452848 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.452894 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:11.777803 (+2324909us) ts_tablet_manager.cc:1151] Deleting tablet 12501fdf2cc5491cb1a06db490e70810
0505 07:54:11.777813 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.797331 (+ 19518us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.851352 (+ 54021us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.883926 (+ 32574us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2324756}
I20260505 07:54:11.887735 2444 raft_consensus.cc:2749] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:11.884491 1181 raft_consensus.cc:359] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.889134 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 12501fdf2cc5491cb1a06db490e70810 (table default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]) successfully deleted
I20260505 07:54:11.889617 1181 raft_consensus.cc:385] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:11.890542 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 142) took 2437 ms. Trace:
I20260505 07:54:11.889834 1181 raft_consensus.cc:740] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:11.890693 2268 ts_tablet_manager.cc:1929] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.893029 2268 log.cc:1199] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/5ad8a79245ad47738d079752e51aed2b
I20260505 07:54:11.893100 1181 consensus_queue.cc:260] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.891388 2447 rpcz_store.cc:276] 0505 07:54:09.453182 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.453227 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:11.888929 (+2435702us) ts_tablet_manager.cc:1151] Deleting tablet a7fe9c7199cc4039ae877dc3d4aab17e
0505 07:54:11.888942 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.890207 (+ 1265us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2435052}
W20260505 07:54:11.894282 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet a7fe9c7199cc4039ae877dc3d4aab17e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7fe9c7199cc4039ae877dc3d4aab17e already in progress: creating tablet
I20260505 07:54:11.894084 1181 ts_tablet_manager.cc:1434] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.010s user 0.000s sys 0.001s
I20260505 07:54:11.896720 2280 raft_consensus.cc:515] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:11.898229 718 leader_election.cc:343] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 54e4f7f895d943639653c8c32724baf2
I20260505 07:54:11.898510 2268 ts_tablet_manager.cc:1950] T 5ad8a79245ad47738d079752e51aed2b P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:11.899713 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:11.899942 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:11.900120 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:11.900318 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:11.900492 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:11.900740 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:11.901057 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:11.901266 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:11.901275 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54e4f7f895d943639653c8c32724baf2" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:11.901420 1047 raft_consensus.cc:3060] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.901439 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:11.901567 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:11.901846 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:11.902318 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:11.905481 2276 ts_tablet_manager.cc:1929] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.905472 1047 raft_consensus.cc:2468] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:11.905908 717 leader_election.cc:304] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:11.906270 2414 raft_consensus.cc:2804] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:11.906388 2414 raft_consensus.cc:697] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:11.906667 2414 consensus_queue.cc:237] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:11.917220 991 consensus_peers.cc:597] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0f2f46d0e7840458cb992ce2a2dbea2. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:11.921463 2276 log.cc:1199] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/067c32ecccb44bca8bf9cc3f6c229326
I20260505 07:54:11.921985 2276 ts_tablet_manager.cc:1950] T 067c32ecccb44bca8bf9cc3f6c229326 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:11.922343 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 75) took 4261 ms. Trace:
I20260505 07:54:11.922477 2276 rpcz_store.cc:276] 0505 07:54:07.661046 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.661252 (+ 206us) service_pool.cc:224] Handling call
0505 07:54:11.837034 (+4175782us) ts_tablet_manager.cc:1151] Deleting tablet 067c32ecccb44bca8bf9cc3f6c229326
0505 07:54:11.837044 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.862158 (+ 25114us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.905410 (+ 43252us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.922327 (+ 16917us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4175670}
I20260505 07:54:11.922885 2276 tablet_replica.cc:333] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:11.923120 2276 raft_consensus.cc:2243] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:11.923178 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 067c32ecccb44bca8bf9cc3f6c229326 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]) successfully deleted
I20260505 07:54:11.923487 2276 raft_consensus.cc:2272] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.924206 2276 ts_tablet_manager.cc:1916] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.902701 1348 catalog_manager.cc:5671] T e79a1da502604d05a62e27f575b8b8f4 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:11.932363 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:11.932971 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
I20260505 07:54:11.933734 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:11.934464 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 42360f8ee4ff47c2af8d287f792207de (table default.Table_b85a2b107c88498e84217252ca6cb947 [id=2c91868ddba84103b9a5c78201842a9e]). Expected version 2 got 1
I20260505 07:54:11.934875 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:11.935346 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:11.936828 2564 raft_consensus.cc:493] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:11.937041 2564 raft_consensus.cc:515] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.938021 2564 leader_election.cc:290] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.938568 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba616702dc6040bb805bb3ffc8a8a871" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:11.938772 781 raft_consensus.cc:2468] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:11.939306 854 leader_election.cc:304] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:11.939774 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba616702dc6040bb805bb3ffc8a8a871" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:11.940249 856 leader_election.cc:343] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.940586 2564 raft_consensus.cc:2804] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:11.940704 2564 raft_consensus.cc:493] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:11.940806 2564 raft_consensus.cc:3060] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.943639 2564 raft_consensus.cc:515] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.944613 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba616702dc6040bb805bb3ffc8a8a871" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:11.944762 781 raft_consensus.cc:3060] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:11.947870 781 raft_consensus.cc:2468] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:11.948359 854 leader_election.cc:304] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:11.948668 2505 raft_consensus.cc:2804] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:11.948779 2505 raft_consensus.cc:697] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:11.949007 2505 consensus_queue.cc:237] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:11.951923 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Bootstrap replaying log segment 1/1 (187.1K/270.2K this segment, stats: ops{read=414 overwritten=0 applied=414 ignored=0} inserts{seen=387 ignored=0} mutations{seen=466 ignored=0} orphaned_commits=0)
I20260505 07:54:11.960632 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:11.961481 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:11.961830 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:11.962042 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
W20260505 07:54:11.962642 1322 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 5006965919824adbbb23d744fb9ff335 consensus state has no leader
I20260505 07:54:11.962670 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:11.963186 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:11.963406 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:11.980733 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba616702dc6040bb805bb3ffc8a8a871" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:11.981226 856 leader_election.cc:343] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:11.982141 2564 leader_election.cc:290] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.988363 2276 ts_tablet_manager.cc:1929] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:11.988556 2276 log.cc:1199] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/cac5064f68d24ebe95b5a0b8e10f463e
I20260505 07:54:11.984319 1348 catalog_manager.cc:5671] T d442119970a345f28de029eb2139f5c1 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:11.988906 1181 tablet_bootstrap.cc:492] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:11.989167 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:11.989542 2447 tablet_replica.cc:333] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:11.989743 2447 raft_consensus.cc:2243] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:11.989886 2276 ts_tablet_manager.cc:1950] T cac5064f68d24ebe95b5a0b8e10f463e P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:11.990173 2447 raft_consensus.cc:2272] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.990200 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 73) took 4329 ms. Trace:
I20260505 07:54:11.990288 2276 rpcz_store.cc:276] 0505 07:54:07.660830 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.661414 (+ 584us) service_pool.cc:224] Handling call
0505 07:54:11.922842 (+4261428us) ts_tablet_manager.cc:1151] Deleting tablet cac5064f68d24ebe95b5a0b8e10f463e
0505 07:54:11.922852 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.967342 (+ 44490us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.988290 (+ 20948us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.990186 (+ 1896us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4261306}
I20260505 07:54:11.990742 1181 tablet_bootstrap.cc:654] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:11.990739 2447 ts_tablet_manager.cc:1916] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.989372 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:11.991583 2280 leader_election.cc:290] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:11.991688 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet cac5064f68d24ebe95b5a0b8e10f463e (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]) successfully deleted
I20260505 07:54:11.994017 2276 tablet_replica.cc:333] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:11.994274 2276 raft_consensus.cc:2243] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:11.994611 2276 raft_consensus.cc:2272] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.995234 2276 ts_tablet_manager.cc:1916] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:11.996459 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:11.996600 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 120) took 2410 ms. Trace:
I20260505 07:54:11.996727 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:11.996672 2268 rpcz_store.cc:276] 0505 07:54:09.585826 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.585886 (+ 60us) service_pool.cc:224] Handling call
0505 07:54:11.806147 (+2220261us) ts_tablet_manager.cc:1151] Deleting tablet 5ad8a79245ad47738d079752e51aed2b
0505 07:54:11.806158 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.853228 (+ 47070us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.890078 (+ 36850us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:11.996584 (+106506us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2219868}
I20260505 07:54:11.996973 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:11.997097 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 121) took 2410 ms. Trace:
I20260505 07:54:11.997190 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 5ad8a79245ad47738d079752e51aed2b (table default.Table_ba3de8258d7e46c1ad3a509b8e00b806 [id=75d1988e3a9e4ff3a8c206a662f55131]) successfully deleted
I20260505 07:54:11.997185 2268 rpcz_store.cc:276] 0505 07:54:09.586514 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.586567 (+ 53us) service_pool.cc:224] Handling call
0505 07:54:11.996961 (+2410394us) ts_tablet_manager.cc:1151] Deleting tablet f2f5b72f58a24db0b58c15b47227bc60
0505 07:54:11.996972 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.997080 (+ 108us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2410201}
I20260505 07:54:11.997432 2268 tablet_replica.cc:333] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:11.997607 2268 raft_consensus.cc:2243] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:11.997792 2268 pending_rounds.cc:70] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:54:11.997829 2268 pending_rounds.cc:77] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555501647462400 op_type: NO_OP noop_request { }
I20260505 07:54:11.997921 2268 raft_consensus.cc:2889] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:11.997989 2268 raft_consensus.cc:2272] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:11.998726 2268 ts_tablet_manager.cc:1916] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.000252 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:12.000330 2447 ts_tablet_manager.cc:1929] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.000581 2447 log.cc:1199] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/1ef4f516e64a44fd8cd33674fb808067
I20260505 07:54:12.000775 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:12.001521 2447 ts_tablet_manager.cc:1950] T 1ef4f516e64a44fd8cd33674fb808067 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:12.002313 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 143) took 2548 ms. Trace:
I20260505 07:54:12.002537 2447 rpcz_store.cc:276] 0505 07:54:09.453476 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.453520 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:11.895145 (+2441625us) ts_tablet_manager.cc:1151] Deleting tablet 1ef4f516e64a44fd8cd33674fb808067
0505 07:54:11.989489 (+ 94344us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:11.994301 (+ 4812us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.000197 (+ 5896us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.002252 (+ 2055us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2441174}
I20260505 07:54:12.003557 2447 tablet_replica.cc:333] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:12.004060 2447 raft_consensus.cc:2243] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.004532 2447 raft_consensus.cc:2272] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.005438 2268 ts_tablet_manager.cc:1929] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.005682 2268 log.cc:1199] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/e010f23c4df647ab863e03c3cfed7898
I20260505 07:54:12.006093 2268 ts_tablet_manager.cc:1950] T e010f23c4df647ab863e03c3cfed7898 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.006387 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 122) took 2415 ms. Trace:
I20260505 07:54:12.006492 2268 rpcz_store.cc:276] 0505 07:54:09.590940 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.590984 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:11.997394 (+2406410us) ts_tablet_manager.cc:1151] Deleting tablet e010f23c4df647ab863e03c3cfed7898
0505 07:54:11.997403 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.002409 (+ 5006us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.005401 (+ 2992us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.006374 (+ 973us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2406201}
I20260505 07:54:12.006845 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 123) took 2415 ms. Trace:
I20260505 07:54:12.006958 2268 rpcz_store.cc:276] 0505 07:54:09.591359 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.591402 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:12.006766 (+2415364us) ts_tablet_manager.cc:1151] Deleting tablet ccdde3e5a13e436d9f2328831e155adb
0505 07:54:12.006774 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.006834 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2415091}
I20260505 07:54:12.007045 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:12.007273 2268 tablet_replica.cc:333] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:12.007510 2268 raft_consensus.cc:2243] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.007368 1348 catalog_manager.cc:5671] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:12.007830 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:12.008147 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 21480c42b73e4bf0ab2453b085a867c0 (table default.Table_925028e721e24ba68f31fe1040d4c2fd [id=0faa6676bab04919984b580299143271]). Expected version 1 got 0
I20260505 07:54:12.008196 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 1ef4f516e64a44fd8cd33674fb808067 (table default.Table_b85a2b107c88498e84217252ca6cb947 [id=2c91868ddba84103b9a5c78201842a9e]) successfully deleted
I20260505 07:54:12.008466 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
W20260505 07:54:12.008666 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f2f5b72f58a24db0b58c15b47227bc60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2f5b72f58a24db0b58c15b47227bc60 already in progress: creating tablet
I20260505 07:54:12.008054 2268 raft_consensus.cc:2272] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.012077 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet e010f23c4df647ab863e03c3cfed7898 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]) successfully deleted
I20260505 07:54:12.014302 1176 tablet_bootstrap.cc:492] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.014600 1176 ts_tablet_manager.cc:1403] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.196s user 0.010s sys 0.010s
I20260505 07:54:12.016263 1176 raft_consensus.cc:359] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.016659 1176 raft_consensus.cc:385] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.016803 1176 raft_consensus.cc:740] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.017174 1176 consensus_queue.cc:260] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.022063 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:12.022921 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:12.023144 1348 catalog_manager.cc:5671] T 2ca12a17fb014374bf7ed764a91e6985 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:12.023713 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
W20260505 07:54:12.023766 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet ccdde3e5a13e436d9f2328831e155adb because the tablet was not found. No further retry: Not found: Tablet not found: ccdde3e5a13e436d9f2328831e155adb
I20260505 07:54:12.023412 2447 ts_tablet_manager.cc:1916] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.024047 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:12.024364 1348 catalog_manager.cc:5671] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:12.024825 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:12.025147 2268 ts_tablet_manager.cc:1916] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.025532 1355 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet f57e14ee70584d258c83ff1b01189713
I20260505 07:54:12.026099 1355 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 154) took 1159 ms. Trace:
I20260505 07:54:12.026300 887 tablet_service.cc:1558] Processing DeleteTablet for tablet f57e14ee70584d258c83ff1b01189713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:11 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:12.026258 1355 rpcz_store.cc:276] 0505 07:54:10.866665 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.115553 (+248888us) service_pool.cc:224] Handling call
0505 07:54:11.171946 (+ 56393us) spinlock_profiling.cc:243] Waited 7.92 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d7b06 00007fd65a46c486 00007fd65a46c1b5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:11.199245 (+ 27299us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:11.199386 (+ 141us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:11.199448 (+ 62us) catalog_manager.cc:2824] Locking tablets
0505 07:54:11.199586 (+ 138us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:11.209750 (+ 10164us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:11.209832 (+ 82us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:12.025484 (+815652us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:12.026047 (+ 563us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:11.200586 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.200710 (+ 124us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.200717 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:11.200722 (+ 5us) tablet.cc:662] Decoding operations
0505 07:54:11.200835 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.200851 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.200862 (+ 11us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:11.200977 (+ 115us) tablet.cc:701] Row locks acquired
0505 07:54:11.200981 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:11.201012 (+ 31us) write_op.cc:270] Start()
0505 07:54:11.201043 (+ 31us) write_op.cc:276] Timestamp: P: 1777967651200946 usec, L: 0
0505 07:54:11.201049 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.201222 (+ 173us) log.cc:844] Serialized 725 byte log entry
0505 07:54:11.208184 (+ 6962us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.208346 (+ 162us) write_op.cc:301] APPLY: starting
0505 07:54:11.208359 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.208401 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.208430 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.208490 (+ 60us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.208523 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:11.208533 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:11.208593 (+ 60us) log.cc:844] Serialized 29 byte log entry
0505 07:54:11.208657 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.208674 (+ 17us) write_op.cc:454] Released schema lock
0505 07:54:11.208682 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:11.208754 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.208759 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":18218240,"child_traces":[["op",{"apply.queue_time_us":123,"apply.run_cpu_time_us":397,"apply.run_wall_time_us":395,"num_ops":2,"prepare.queue_time_us":243,"prepare.run_cpu_time_us":740,"prepare.run_wall_time_us":740,"raft.queue_time_us":1021,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":7118}]]}
I20260505 07:54:12.028748 1176 ts_tablet_manager.cc:1434] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.014s user 0.001s sys 0.001s
I20260505 07:54:12.031823 1176 tablet_bootstrap.cc:492] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.032083 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet f57e14ee70584d258c83ff1b01189713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:11 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:12.036972 2276 ts_tablet_manager.cc:1929] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.037209 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet b0d966bcd7f1419cbd090afc185ae78a
I20260505 07:54:12.038110 1181 tablet_bootstrap.cc:492] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.038017 887 tablet_service.cc:1558] Processing DeleteTablet for tablet b0d966bcd7f1419cbd090afc185ae78a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:11 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:12.038888 2276 log.cc:1199] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/78a6470720984761a8606abeafbef992
I20260505 07:54:12.039283 2276 ts_tablet_manager.cc:1950] T 78a6470720984761a8606abeafbef992 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:12.040083 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 76) took 4373 ms. Trace:
I20260505 07:54:12.040345 2089 raft_consensus.cc:493] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.040552 2276 rpcz_store.cc:276] 0505 07:54:07.666238 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.666409 (+ 171us) service_pool.cc:224] Handling call
0505 07:54:11.993961 (+4327552us) ts_tablet_manager.cc:1151] Deleting tablet 78a6470720984761a8606abeafbef992
0505 07:54:11.993973 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.025542 (+ 31569us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.036922 (+ 11380us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.040065 (+ 3143us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4327410}
I20260505 07:54:12.040635 2089 raft_consensus.cc:515] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.041730 2276 tablet_replica.cc:333] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
W20260505 07:54:12.042579 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 consensus state has no leader
I20260505 07:54:12.042613 1350 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 156) took 1087 ms. Trace:
I20260505 07:54:12.043604 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 78a6470720984761a8606abeafbef992 (table default.Table_5cd903011cdc44a392680507ed9088c8 [id=a3c321e46ecd4b07809a9c663305afb7]) successfully deleted
I20260505 07:54:12.044791 1181 ts_tablet_manager.cc:1403] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.056s user 0.003s sys 0.006s
I20260505 07:54:12.045943 1181 raft_consensus.cc:359] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.046125 1181 raft_consensus.cc:385] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.046181 1181 raft_consensus.cc:740] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.046450 1181 consensus_queue.cc:260] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.046846 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:12.047024 1181 ts_tablet_manager.cc:1434] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:12.047431 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet b0d966bcd7f1419cbd090afc185ae78a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:11 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:12.047712 1351 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet b8a85b65b1904bd6b793af48f71ed726
I20260505 07:54:12.047793 1181 tablet_bootstrap.cc:492] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.049242 887 tablet_service.cc:1558] Processing DeleteTablet for tablet b8a85b65b1904bd6b793af48f71ed726 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:11 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:12.049942 2276 raft_consensus.cc:2243] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.050127 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f121fdd001ec4ea38ccd24a6afd9e51c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.051280 2505 raft_consensus.cc:493] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.051710 1351 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 152) took 1183 ms. Trace:
I20260505 07:54:12.051645 2505 raft_consensus.cc:515] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:12.052392 1352 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 139d5a2ab9594f6ea6042c1c4ba8a46a consensus state has no leader
W20260505 07:54:12.052474 1357 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f36a0f68fc2b4adda04ec3b9db5f2d8e consensus state has no leader
I20260505 07:54:12.050158 2276 raft_consensus.cc:2272] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.052745 1181 tablet_bootstrap.cc:654] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:12.053443 1354 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 180157193a184add848ba67ca8042109 consensus state has no leader
I20260505 07:54:12.053479 1349 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 151) took 1180 ms. Trace:
I20260505 07:54:12.053551 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df1b62a30b2842b897a04c793bb5e7ce" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.053609 1354 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 155) took 1116 ms. Trace:
I20260505 07:54:12.053746 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet b8a85b65b1904bd6b793af48f71ed726 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:11 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:12.054061 1354 rpcz_store.cc:276] 0505 07:54:10.936788 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.134773 (+197985us) service_pool.cc:224] Handling call
0505 07:54:11.172271 (+ 37498us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d8da6 00007fd65a46e146 00007fd65a46de75 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:11.172591 (+ 320us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:11.172838 (+ 247us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:11.198983 (+ 26145us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:11.199093 (+ 110us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:11.209266 (+ 10173us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:12.053556 (+844290us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:11.199742 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.199877 (+ 135us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.199883 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:11.199885 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:11.199981 (+ 96us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.199996 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.200001 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:11.200071 (+ 70us) tablet.cc:701] Row locks acquired
0505 07:54:11.200074 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:11.200118 (+ 44us) write_op.cc:270] Start()
0505 07:54:11.200155 (+ 37us) write_op.cc:276] Timestamp: P: 1777967651200048 usec, L: 0
0505 07:54:11.200161 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.200409 (+ 248us) log.cc:844] Serialized 405 byte log entry
0505 07:54:11.208114 (+ 7705us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.208335 (+ 221us) write_op.cc:301] APPLY: starting
0505 07:54:11.208353 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.208410 (+ 57us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.208480 (+ 70us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.208574 (+ 94us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.208613 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:11.208624 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:11.208765 (+ 141us) log.cc:844] Serialized 23 byte log entry
0505 07:54:11.209109 (+ 344us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.209122 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:11.209127 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:11.209191 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.209194 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":60909312,"child_traces":[["op",{"apply.queue_time_us":96,"apply.run_cpu_time_us":645,"apply.run_wall_time_us":868,"num_ops":1,"prepare.queue_time_us":47,"prepare.run_cpu_time_us":797,"prepare.run_wall_time_us":797,"raft.queue_time_us":1478,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":276,"replication_time_us":7924}]]}
I20260505 07:54:12.057978 2650 raft_consensus.cc:493] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.058157 1352 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 157) took 1101 ms. Trace:
I20260505 07:54:12.058115 2650 raft_consensus.cc:515] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.058241 1352 rpcz_store.cc:276] 0505 07:54:10.952626 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.263930 (+311304us) service_pool.cc:224] Handling call
0505 07:54:11.264356 (+ 426us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:11.264581 (+ 225us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:11.264777 (+ 196us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:11.264886 (+ 109us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:11.270722 (+ 5836us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:12.053727 (+783005us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:11.265419 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.265527 (+ 108us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.265534 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:11.265537 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:11.265642 (+ 105us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.265654 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.265660 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:11.265720 (+ 60us) tablet.cc:701] Row locks acquired
0505 07:54:11.265723 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:11.265758 (+ 35us) write_op.cc:270] Start()
0505 07:54:11.265788 (+ 30us) write_op.cc:276] Timestamp: P: 1777967651265734 usec, L: 0
0505 07:54:11.265792 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.265969 (+ 177us) log.cc:844] Serialized 405 byte log entry
0505 07:54:11.269437 (+ 3468us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.269859 (+ 422us) write_op.cc:301] APPLY: starting
0505 07:54:11.269875 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.269916 (+ 41us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.269940 (+ 24us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.269990 (+ 50us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.270020 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:11.270027 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:11.270121 (+ 94us) log.cc:844] Serialized 23 byte log entry
0505 07:54:11.270180 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.270199 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:11.270206 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:11.270278 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.270282 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":322,"child_traces":[["op",{"apply.queue_time_us":336,"apply.run_cpu_time_us":418,"apply.run_wall_time_us":418,"num_ops":1,"prepare.queue_time_us":43,"prepare.run_cpu_time_us":657,"prepare.run_wall_time_us":658,"raft.queue_time_us":470,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":240,"replication_time_us":3618}]]}
I20260505 07:54:12.057920 2089 leader_election.cc:290] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.052834 1350 rpcz_store.cc:276] 0505 07:54:10.950562 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.229697 (+279135us) service_pool.cc:224] Handling call
0505 07:54:11.230134 (+ 437us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:11.230277 (+ 143us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:11.230338 (+ 61us) catalog_manager.cc:2824] Locking tablets
0505 07:54:11.314722 (+ 84384us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:11.320357 (+ 5635us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:11.320438 (+ 81us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:12.037160 (+716722us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:12.037706 (+ 546us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:11.315455 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.315626 (+ 171us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.315633 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:11.315635 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:11.315757 (+ 122us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.315769 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.315773 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:11.315889 (+ 116us) tablet.cc:701] Row locks acquired
0505 07:54:11.315892 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:11.315928 (+ 36us) write_op.cc:270] Start()
0505 07:54:11.315957 (+ 29us) write_op.cc:276] Timestamp: P: 1777967651315898 usec, L: 0
0505 07:54:11.315961 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.316118 (+ 157us) log.cc:844] Serialized 756 byte log entry
0505 07:54:11.319094 (+ 2976us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.319837 (+ 743us) write_op.cc:301] APPLY: starting
0505 07:54:11.319850 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.319890 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.319916 (+ 26us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.319977 (+ 61us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.320013 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:11.320022 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:11.320087 (+ 65us) log.cc:844] Serialized 29 byte log entry
0505 07:54:11.320143 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.320160 (+ 17us) write_op.cc:454] Released schema lock
0505 07:54:11.320166 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:11.320228 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.320232 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":677,"apply.run_cpu_time_us":389,"apply.run_wall_time_us":398,"num_ops":2,"prepare.queue_time_us":42,"prepare.run_cpu_time_us":760,"prepare.run_wall_time_us":759,"raft.queue_time_us":360,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":223,"replication_time_us":3105}]]}
I20260505 07:54:12.059834 1357 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 153) took 1188 ms. Trace:
I20260505 07:54:12.059930 1357 rpcz_store.cc:276] 0505 07:54:10.865055 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.085159 (+220104us) service_pool.cc:224] Handling call
0505 07:54:11.173602 (+ 88443us) spinlock_profiling.cc:243] Waited 87.9 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d8da6 00007fd65a46e146 00007fd65a46de75 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:11.199960 (+ 26358us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:11.200153 (+ 193us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:11.200303 (+ 150us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:11.200389 (+ 86us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:11.209227 (+ 8838us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:12.053410 (+844183us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:11.201353 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.201461 (+ 108us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.201468 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:11.201472 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:11.201558 (+ 86us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.201581 (+ 23us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.201588 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:11.201645 (+ 57us) tablet.cc:701] Row locks acquired
0505 07:54:11.201649 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:11.201675 (+ 26us) write_op.cc:270] Start()
0505 07:54:11.201703 (+ 28us) write_op.cc:276] Timestamp: P: 1777967651201610 usec, L: 0
0505 07:54:11.201708 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.201865 (+ 157us) log.cc:844] Serialized 405 byte log entry
0505 07:54:11.208238 (+ 6373us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.208839 (+ 601us) write_op.cc:301] APPLY: starting
0505 07:54:11.208849 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.208875 (+ 26us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.208895 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.208931 (+ 36us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.208967 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:11.208978 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:11.209044 (+ 66us) log.cc:844] Serialized 23 byte log entry
0505 07:54:11.209126 (+ 82us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.209142 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:11.209148 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:11.209247 (+ 99us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.209253 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":202063616,"child_traces":[["op",{"apply.queue_time_us":560,"apply.run_cpu_time_us":365,"apply.run_wall_time_us":365,"num_ops":1,"prepare.queue_time_us":573,"prepare.run_cpu_time_us":595,"prepare.run_wall_time_us":595,"raft.queue_time_us":405,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"replication_time_us":6515,"spinlock_wait_cycles":30976}]]}
I20260505 07:54:12.053592 1349 rpcz_store.cc:276] 0505 07:54:10.862592 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.042615 (+180023us) service_pool.cc:224] Handling call
0505 07:54:11.042990 (+ 375us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:11.043193 (+ 203us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:11.043362 (+ 169us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:11.043435 (+ 73us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:11.131864 (+ 88429us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:12.042734 (+910870us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:11.048968 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.049055 (+ 87us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.049060 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:11.049062 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:11.049137 (+ 75us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.049145 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.049149 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:11.049196 (+ 47us) tablet.cc:701] Row locks acquired
0505 07:54:11.049198 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:11.049224 (+ 26us) write_op.cc:270] Start()
0505 07:54:11.049247 (+ 23us) write_op.cc:276] Timestamp: P: 1777967651049175 usec, L: 0
0505 07:54:11.049251 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.049394 (+ 143us) log.cc:844] Serialized 405 byte log entry
0505 07:54:11.130581 (+ 81187us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.130681 (+ 100us) write_op.cc:301] APPLY: starting
0505 07:54:11.130698 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.130762 (+ 64us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.131339 (+ 577us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.131396 (+ 57us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.131434 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:11.131448 (+ 14us) write_op.cc:312] APPLY: finished
0505 07:54:11.131533 (+ 85us) log.cc:844] Serialized 23 byte log entry
0505 07:54:11.131624 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.131659 (+ 35us) write_op.cc:454] Released schema lock
0505 07:54:11.131667 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:11.131756 (+ 89us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.131763 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":45,"apply.run_cpu_time_us":537,"apply.run_wall_time_us":1067,"num_ops":1,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":493,"prepare.run_wall_time_us":494,"replication_time_us":81302}]]}
I20260505 07:54:12.062917 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_f38a3721f0774a7bbc71a46b8107ee83"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:12.063606 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f38a3721f0774a7bbc71a46b8107ee83 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:12.063905 757 tablet_service.cc:1558] Processing DeleteTablet for tablet b0d966bcd7f1419cbd090afc185ae78a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:11 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.064236 757 tablet_service.cc:1558] Processing DeleteTablet for tablet b8a85b65b1904bd6b793af48f71ed726 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:11 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.065462 751 tablet_service.cc:1558] Processing DeleteTablet for tablet f57e14ee70584d258c83ff1b01189713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:11 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.066159 2650 leader_election.cc:290] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.066730 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e99f96a753047348371bee5c022684e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.068545 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_00e64df3348c430399b8bb1afc7f44ee" } new_table_name: "default.Table_b20b461c86aa4e52b4580432defb5b96" modify_external_catalogs: true
I20260505 07:54:12.069092 1350 master_service.cc:954] ReplaceTablet: received request to replace tablet f840d5f58161468385d5c4f116b78c3d from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:12.069504 1352 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_4ab0ac01b8bf49ef96683dca41dfadd4" } modify_external_catalogs: true
I20260505 07:54:12.051819 1351 rpcz_store.cc:276] 0505 07:54:10.864902 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.084998 (+220096us) service_pool.cc:224] Handling call
0505 07:54:11.161032 (+ 76034us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d7b06 00007fd65a46c486 00007fd65a46c1b5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:11.173873 (+ 12841us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:11.199823 (+ 25950us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:11.199870 (+ 47us) catalog_manager.cc:2824] Locking tablets
0505 07:54:11.319160 (+119290us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:11.327779 (+ 8619us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:11.327862 (+ 83us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:12.038645 (+710783us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:12.048229 (+ 9584us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:11.320235 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.320329 (+ 94us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.320332 (+ 3us) write_op.cc:435] Acquired schema lock
0505 07:54:11.320334 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:11.320423 (+ 89us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.320435 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.320440 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:11.320531 (+ 91us) tablet.cc:701] Row locks acquired
0505 07:54:11.320533 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:11.320558 (+ 25us) write_op.cc:270] Start()
0505 07:54:11.320595 (+ 37us) write_op.cc:276] Timestamp: P: 1777967651320531 usec, L: 0
0505 07:54:11.320600 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.320728 (+ 128us) log.cc:844] Serialized 725 byte log entry
0505 07:54:11.325280 (+ 4552us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.325933 (+ 653us) write_op.cc:301] APPLY: starting
0505 07:54:11.325947 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.325982 (+ 35us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.326011 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.326075 (+ 64us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.326103 (+ 28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:11.326110 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:11.326163 (+ 53us) log.cc:844] Serialized 29 byte log entry
0505 07:54:11.326209 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.326222 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:11.326227 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:11.326300 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.326304 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":130605824,"child_traces":[["op",{"apply.queue_time_us":600,"apply.run_cpu_time_us":363,"apply.run_wall_time_us":363,"num_ops":2,"prepare.queue_time_us":456,"prepare.run_cpu_time_us":585,"prepare.run_wall_time_us":585,"raft.queue_time_us":151,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":226,"replication_time_us":4612}]]}
I20260505 07:54:12.070312 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f121fdd001ec4ea38ccd24a6afd9e51c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:12.071022 1351 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_be7562a92105457caa8d9ad5d34012b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:12.072283 1351 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_be7562a92105457caa8d9ad5d34012b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:12.073500 1176 tablet_bootstrap.cc:654] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.073942 2268 ts_tablet_manager.cc:1929] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.074096 2268 log.cc:1199] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/7585f00119e14006990ed551667b4486
I20260505 07:54:12.074674 717 leader_election.cc:304] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:12.075229 2673 raft_consensus.cc:2749] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.076239 2268 ts_tablet_manager.cc:1950] T 7585f00119e14006990ed551667b4486 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.076606 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 124) took 2484 ms. Trace:
I20260505 07:54:12.077304 2268 rpcz_store.cc:276] 0505 07:54:09.591834 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.591877 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:12.007235 (+2415358us) ts_tablet_manager.cc:1151] Deleting tablet 7585f00119e14006990ed551667b4486
0505 07:54:12.007244 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.030444 (+ 23200us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.073876 (+ 43432us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.076587 (+ 2711us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2415159}
I20260505 07:54:12.078155 2268 tablet_replica.cc:333] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:12.078441 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 7585f00119e14006990ed551667b4486 (table default.Table_c61fca437ddb413e934595a23b52f40f [id=a14990bc69324c54a3c7e2b7d6116d11]) successfully deleted
I20260505 07:54:12.079239 2268 raft_consensus.cc:2243] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.079448 2268 raft_consensus.cc:2272] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.080089 2268 ts_tablet_manager.cc:1916] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:12.083860 988 leader_election.cc:343] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.088531 2632 tablet.cc:1724] T 90a13ca612f241fc97ea12124677a0b1 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:12.089407 2596 tablet.cc:1724] T 90a13ca612f241fc97ea12124677a0b1 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:12.108453 2268 ts_tablet_manager.cc:1929] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.117331 2268 log.cc:1199] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/3d6494590052459485059f5dce46f34f
I20260505 07:54:12.117792 2268 ts_tablet_manager.cc:1950] T 3d6494590052459485059f5dce46f34f P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
W20260505 07:54:12.054083 854 leader_election.cc:343] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:12.120040 854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 65959.6us
I20260505 07:54:12.120424 2444 raft_consensus.cc:493] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.120539 2444 raft_consensus.cc:515] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.068526 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e99f96a753047348371bee5c022684e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.121033 912 raft_consensus.cc:2468] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:12.121165 2444 leader_election.cc:290] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.121662 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3a49b59ed6485f970074391f932482" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.090608 2652 tablet.cc:1724] T 90a13ca612f241fc97ea12124677a0b1 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:12.122022 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3a49b59ed6485f970074391f932482" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:12.122190 718 consensus_peers.cc:597] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 067c32ecccb44bca8bf9cc3f6c229326. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:12.122485 856 leader_election.cc:343] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:12.122766 854 leader_election.cc:343] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.122826 2505 leader_election.cc:290] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.122849 991 leader_election.cc:304] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.122866 854 leader_election.cc:304] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.123293 2505 raft_consensus.cc:2749] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.123549 2307 raft_consensus.cc:2804] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.123651 2307 raft_consensus.cc:493] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.123731 2307 raft_consensus.cc:3060] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.123948 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df1b62a30b2842b897a04c793bb5e7ce" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:12.124119 1051 raft_consensus.cc:2468] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:12.124562 856 leader_election.cc:304] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.125442 2505 raft_consensus.cc:2804] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.125280 2276 ts_tablet_manager.cc:1916] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.125504 2505 raft_consensus.cc:493] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.125566 2505 raft_consensus.cc:3060] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.127053 2307 raft_consensus.cc:515] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.128423 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e99f96a753047348371bee5c022684e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:12.128645 991 consensus_peers.cc:597] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 17327cee69a24e0cb493ff55f948bf6c. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:12.129010 988 leader_election.cc:343] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.130923 2505 raft_consensus.cc:515] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.131834 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df1b62a30b2842b897a04c793bb5e7ce" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:12.132176 2505 leader_election.cc:290] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.114367 1172 tablet_bootstrap.cc:492] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:12.132470 1172 ts_tablet_manager.cc:1403] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.285s user 0.032s sys 0.049s
W20260505 07:54:12.132694 854 leader_election.cc:343] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.133407 1172 raft_consensus.cc:359] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.133630 1172 raft_consensus.cc:385] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.133737 1172 raft_consensus.cc:740] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.134025 1172 consensus_queue.cc:260] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.134684 1172 ts_tablet_manager.cc:1434] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:12.135124 1172 tablet_bootstrap.cc:492] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:12.136193 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e99f96a753047348371bee5c022684e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:12.136350 921 raft_consensus.cc:3060] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.136899 1172 tablet_bootstrap.cc:654] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.138669 921 raft_consensus.cc:2468] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:12.139170 991 leader_election.cc:304] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.139415 2405 raft_consensus.cc:2804] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:12.139525 2405 raft_consensus.cc:697] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:12.139760 2405 consensus_queue.cc:237] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.139999 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df1b62a30b2842b897a04c793bb5e7ce" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:12.140122 1051 raft_consensus.cc:3060] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.142855 1051 raft_consensus.cc:2468] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:12.143324 856 leader_election.cc:304] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.146142 2307 leader_election.cc:290] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.146392 2505 raft_consensus.cc:2804] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:12.146626 2505 raft_consensus.cc:697] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:12.146917 2505 consensus_queue.cc:237] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.152750 1176 tablet_bootstrap.cc:492] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.152937 1176 ts_tablet_manager.cc:1403] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.122s user 0.002s sys 0.002s
I20260505 07:54:12.154970 1176 raft_consensus.cc:359] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:12.155203 1176 raft_consensus.cc:385] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.155311 1176 raft_consensus.cc:740] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.155542 1176 consensus_queue.cc:260] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:12.155701 2307 consensus_queue.cc:1048] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:12.156128 1176 ts_tablet_manager.cc:1434] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:12.156390 1176 tablet_bootstrap.cc:492] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.165727 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 125) took 2573 ms. Trace:
I20260505 07:54:12.165875 2268 rpcz_store.cc:276] 0505 07:54:09.592233 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.592328 (+ 95us) service_pool.cc:224] Handling call
0505 07:54:12.078113 (+2485785us) ts_tablet_manager.cc:1151] Deleting tablet 3d6494590052459485059f5dce46f34f
0505 07:54:12.078123 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.105477 (+ 27354us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.108405 (+ 2928us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.165667 (+ 57262us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2485593}
I20260505 07:54:12.166302 2268 tablet_replica.cc:333] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:12.166369 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 3d6494590052459485059f5dce46f34f (table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f]) successfully deleted
I20260505 07:54:12.166540 2268 raft_consensus.cc:2243] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:12.166915 2268 raft_consensus.cc:2272] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.167476 2268 ts_tablet_manager.cc:1916] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.172982 2447 ts_tablet_manager.cc:1929] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.173097 2447 log.cc:1199] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/17327cee69a24e0cb493ff55f948bf6c
I20260505 07:54:12.173534 771 raft_consensus.cc:3060] T e1a403e4708e4725899c1cd5b9fb0e8b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.176822 2302 consensus_queue.cc:1048] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:12.178179 1172 tablet_bootstrap.cc:492] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:12.179090 1172 ts_tablet_manager.cc:1403] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.000s
I20260505 07:54:12.183945 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 93d878ae0ddb42319560f74e0a490321 with delete_type TABLET_DATA_DELETED (replaced by tablet bc9ae6404dd64bd8bf4898768e94e4f1) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.184211 1172 raft_consensus.cc:359] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.184376 1172 raft_consensus.cc:385] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.184433 1172 raft_consensus.cc:740] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.184478 1181 tablet_bootstrap.cc:492] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.184623 1181 ts_tablet_manager.cc:1403] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.137s user 0.001s sys 0.013s
I20260505 07:54:12.184646 1172 consensus_queue.cc:260] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.185448 1181 raft_consensus.cc:359] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.185710 1181 raft_consensus.cc:385] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.185813 1181 raft_consensus.cc:740] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.186045 1181 consensus_queue.cc:260] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.186615 1181 ts_tablet_manager.cc:1434] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:12.186944 1181 tablet_bootstrap.cc:492] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.187729 1172 ts_tablet_manager.cc:1434] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260505 07:54:12.188009 1172 tablet_bootstrap.cc:492] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:12.173537 2447 ts_tablet_manager.cc:1950] T 17327cee69a24e0cb493ff55f948bf6c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:12.188441 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 144) took 2734 ms. Trace:
I20260505 07:54:12.188524 2447 rpcz_store.cc:276] 0505 07:54:09.453774 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.453820 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:12.003421 (+2549601us) ts_tablet_manager.cc:1151] Deleting tablet 17327cee69a24e0cb493ff55f948bf6c
0505 07:54:12.003474 (+ 53us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.076992 (+ 73518us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.172912 (+ 95920us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.188425 (+ 15513us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2549444}
I20260505 07:54:12.188853 2447 tablet_replica.cc:333] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:12.189034 2447 raft_consensus.cc:2243] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.189203 2447 raft_consensus.cc:2272] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.189314 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 17327cee69a24e0cb493ff55f948bf6c (table default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]) successfully deleted
W20260505 07:54:12.189558 717 consensus_peers.cc:597] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b57c1c618484d5d86a846fb982a2240. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:12.190250 1181 tablet_bootstrap.cc:654] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.191941 2447 ts_tablet_manager.cc:1916] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.193641 2268 ts_tablet_manager.cc:1929] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.194340 1172 tablet_bootstrap.cc:654] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.201764 2268 log.cc:1199] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/1838f89dc14a47f99f95f4cc5fdefd70
I20260505 07:54:12.202252 2268 ts_tablet_manager.cc:1950] T 1838f89dc14a47f99f95f4cc5fdefd70 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.203166 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 126) took 2601 ms. Trace:
I20260505 07:54:12.203342 2268 rpcz_store.cc:276] 0505 07:54:09.601345 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.601387 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:12.166224 (+2564837us) ts_tablet_manager.cc:1151] Deleting tablet 1838f89dc14a47f99f95f4cc5fdefd70
0505 07:54:12.166232 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.170600 (+ 4368us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.193546 (+ 22946us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.203125 (+ 9579us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2564648}
I20260505 07:54:12.205180 1176 tablet_bootstrap.cc:654] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.205519 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 1838f89dc14a47f99f95f4cc5fdefd70 (table default.Table_d75faf2076d74755af036055448af75a [id=28145ddd58584081b8c48f63009dc958]) successfully deleted
I20260505 07:54:12.218570 1176 tablet_bootstrap.cc:492] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.218704 1176 ts_tablet_manager.cc:1403] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.062s user 0.002s sys 0.004s
I20260505 07:54:12.219399 1176 raft_consensus.cc:359] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:12.223438 2257 log.cc:927] Time spent T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260505 07:54:12.219565 1176 raft_consensus.cc:385] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.224423 1176 raft_consensus.cc:740] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.224712 1176 consensus_queue.cc:260] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.225312 1176 ts_tablet_manager.cc:1434] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260505 07:54:12.225575 1176 tablet_bootstrap.cc:492] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.227104 1176 tablet_bootstrap.cc:654] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.233500 2447 ts_tablet_manager.cc:1929] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.233618 2447 log.cc:1199] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/bde59d52fd234343b1ca13a7d4aae6bf
I20260505 07:54:12.233994 2447 ts_tablet_manager.cc:1950] T bde59d52fd234343b1ca13a7d4aae6bf P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:12.234442 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 145) took 2780 ms. Trace:
I20260505 07:54:12.234536 2447 rpcz_store.cc:276] 0505 07:54:09.454083 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.454126 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:12.188800 (+2734674us) ts_tablet_manager.cc:1151] Deleting tablet bde59d52fd234343b1ca13a7d4aae6bf
0505 07:54:12.188808 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.208537 (+ 19729us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.233442 (+ 24905us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.234423 (+ 981us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2734545}
I20260505 07:54:12.234952 2674 raft_consensus.cc:493] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.234999 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet bde59d52fd234343b1ca13a7d4aae6bf (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]) successfully deleted
I20260505 07:54:12.235081 2674 raft_consensus.cc:515] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:12.235774 2674 leader_election.cc:290] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:12.236719 718 leader_election.cc:343] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 7219f9896fd94187ae714635077110af
W20260505 07:54:12.237613 717 leader_election.cc:343] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 7219f9896fd94187ae714635077110af
I20260505 07:54:12.237706 717 leader_election.cc:304] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:12.237732 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 127) took 2636 ms. Trace:
I20260505 07:54:12.237805 2268 rpcz_store.cc:276] 0505 07:54:09.601735 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.601775 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:12.214971 (+2613196us) ts_tablet_manager.cc:1151] Deleting tablet 840eebf9e7404bf7807b7e51996957e5
0505 07:54:12.237621 (+ 22650us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.237687 (+ 66us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2613007}
I20260505 07:54:12.238121 2674 raft_consensus.cc:2749] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.238194 2268 tablet_replica.cc:333] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
W20260505 07:54:12.238183 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 840eebf9e7404bf7807b7e51996957e5 because the tablet was not found. No further retry: Not found: Tablet not found: 840eebf9e7404bf7807b7e51996957e5
I20260505 07:54:12.238368 2268 raft_consensus.cc:2243] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:12.238657 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.238735 2268 raft_consensus.cc:2272] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.239147 2447 tablet_replica.cc:333] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:12.239365 2447 raft_consensus.cc:2243] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:12.239428 2268 ts_tablet_manager.cc:1916] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.243223 2447 raft_consensus.cc:2272] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.244182 2302 raft_consensus.cc:493] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.244305 2302 raft_consensus.cc:515] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.244765 2447 ts_tablet_manager.cc:1916] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.245347 1352 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_4ab0ac01b8bf49ef96683dca41dfadd4" } modify_external_catalogs: true
I20260505 07:54:12.252311 1181 tablet_bootstrap.cc:492] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.253063 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f200abef13ac42698a9b96fc8af82ff0" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.253253 921 raft_consensus.cc:2468] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:12.258256 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f200abef13ac42698a9b96fc8af82ff0" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.258692 771 raft_consensus.cc:2468] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:12.262095 751 tablet_service.cc:1558] Processing DeleteTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e08cf03ea0b495a94c6566ea6c8caad) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.252635 2302 leader_election.cc:290] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.253646 991 leader_election.cc:304] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:12.272619 2302 raft_consensus.cc:2804] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.272709 2302 raft_consensus.cc:493] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.272768 2302 raft_consensus.cc:3060] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.254513 2268 ts_tablet_manager.cc:1929] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.274953 2268 log.cc:1199] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/1602ccbae24544c4a6cf14ea073880b4
I20260505 07:54:12.275211 2302 raft_consensus.cc:515] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.275296 2268 ts_tablet_manager.cc:1950] T 1602ccbae24544c4a6cf14ea073880b4 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.275955 2302 leader_election.cc:290] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.275993 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 129) took 2672 ms. Trace:
I20260505 07:54:12.276154 1181 ts_tablet_manager.cc:1403] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.089s user 0.001s sys 0.004s
I20260505 07:54:12.276088 2632 tablet.cc:1724] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:12.276050 2268 rpcz_store.cc:276] 0505 07:54:09.603917 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.603978 (+ 61us) service_pool.cc:224] Handling call
0505 07:54:12.238152 (+2634174us) ts_tablet_manager.cc:1151] Deleting tablet 1602ccbae24544c4a6cf14ea073880b4
0505 07:54:12.238161 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.246755 (+ 8594us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.254455 (+ 7700us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.275981 (+ 21526us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2633958}
I20260505 07:54:12.276613 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 130) took 2669 ms. Trace:
I20260505 07:54:12.276634 2652 tablet.cc:1724] T dd3fc8a9d9f04636b1f31bb88411c1da P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:12.276682 2268 rpcz_store.cc:276] 0505 07:54:09.607333 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.607376 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:12.276487 (+2669111us) ts_tablet_manager.cc:1151] Deleting tablet 6546745e894545ff9c5f91bcc6e1a41e
0505 07:54:12.276496 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.276599 (+ 103us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2668901}
I20260505 07:54:12.277002 2268 tablet_replica.cc:333] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:12.277165 2268 raft_consensus.cc:2243] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:12.277199 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 1602ccbae24544c4a6cf14ea073880b4 (table default.Table_e9ba8a72d90d4a3da2c06b684fe35033 [id=5c8387e5e7f84731afc62bbea6475a85]) successfully deleted
I20260505 07:54:12.277437 2268 raft_consensus.cc:2272] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:12.277446 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 6546745e894545ff9c5f91bcc6e1a41e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6546745e894545ff9c5f91bcc6e1a41e already in progress: creating tablet
I20260505 07:54:12.277866 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f200abef13ac42698a9b96fc8af82ff0" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:12.277949 2268 ts_tablet_manager.cc:1916] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.278024 772 raft_consensus.cc:3060] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.279208 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f200abef13ac42698a9b96fc8af82ff0" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:12.279645 907 raft_consensus.cc:3060] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.280722 772 raft_consensus.cc:2468] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:12.281087 988 leader_election.cc:304] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:12.281399 2302 raft_consensus.cc:2804] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:12.281538 2302 raft_consensus.cc:697] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:12.281837 2302 consensus_queue.cc:237] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.285399 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.285653 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 12501fdf2cc5491cb1a06db490e70810 with delete_type TABLET_DATA_DELETED (replaced by tablet d622612bdc404727a27b83769bf39738) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.285995 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 with delete_type TABLET_DATA_DELETED (replaced by tablet 4af9def97a454f3f9b63d637db42c110) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.286109 1172 tablet_bootstrap.cc:492] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:12.286214 1172 ts_tablet_manager.cc:1403] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.098s user 0.005s sys 0.011s
I20260505 07:54:12.286860 1172 raft_consensus.cc:359] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.287004 1172 raft_consensus.cc:385] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.287055 1172 raft_consensus.cc:740] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.287238 1172 consensus_queue.cc:260] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.287703 1172 ts_tablet_manager.cc:1434] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260505 07:54:12.288179 1172 tablet_bootstrap.cc:492] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:12.289876 1172 tablet_bootstrap.cc:654] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.290088 2672 raft_consensus.cc:493] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.290199 2672 raft_consensus.cc:515] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.290759 2672 leader_election.cc:290] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.291211 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76939b7978ee4f7ea0ad29312de2a31a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.291394 921 raft_consensus.cc:2393] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:12.291781 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "76939b7978ee4f7ea0ad29312de2a31a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:12.292364 718 leader_election.cc:304] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:12.292349 1181 raft_consensus.cc:359] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.292484 1181 raft_consensus.cc:385] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.292529 1181 raft_consensus.cc:740] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.292709 1181 consensus_queue.cc:260] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.293183 1181 ts_tablet_manager.cc:1434] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.017s user 0.001s sys 0.001s
I20260505 07:54:12.294514 2672 raft_consensus.cc:3060] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.297159 2672 raft_consensus.cc:2749] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.299082 2596 tablet.cc:1724] T dd3fc8a9d9f04636b1f31bb88411c1da P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260505 07:54:12.299207 1353 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 17b4e886bebc4ab6a3c95290d18b4229 consensus state has no leader
I20260505 07:54:12.300063 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 17327cee69a24e0cb493ff55f948bf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.300541 751 tablet_service.cc:1558] Processing DeleteTablet for tablet bde59d52fd234343b1ca13a7d4aae6bf with delete_type TABLET_DATA_DELETED (replaced by tablet 3a73dbb979344805876efbafbf1b2985) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.300736 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 with delete_type TABLET_DATA_DELETED (replaced by tablet 42360f8ee4ff47c2af8d287f792207de) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.304574 1181 tablet_bootstrap.cc:492] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.305934 1181 tablet_bootstrap.cc:654] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.307389 2268 ts_tablet_manager.cc:1929] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.308665 2447 ts_tablet_manager.cc:1929] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.308776 2447 log.cc:1199] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/23fa401bb4634022bca3c6b23421e4d5
I20260505 07:54:12.308893 2268 log.cc:1199] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/727cacd242844455915bd9963d2db19f
I20260505 07:54:12.309187 2447 ts_tablet_manager.cc:1950] T 23fa401bb4634022bca3c6b23421e4d5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:12.309496 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 146) took 2855 ms. Trace:
I20260505 07:54:12.309525 2268 ts_tablet_manager.cc:1950] T 727cacd242844455915bd9963d2db19f P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.309576 2447 rpcz_store.cc:276] 0505 07:54:09.454362 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.454405 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:12.239103 (+2784698us) ts_tablet_manager.cc:1151] Deleting tablet 23fa401bb4634022bca3c6b23421e4d5
0505 07:54:12.239113 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.248137 (+ 9024us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.308613 (+ 60476us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.309482 (+ 869us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2784575}
I20260505 07:54:12.310078 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 23fa401bb4634022bca3c6b23421e4d5 (table default.Table_0562bf620b5f46ecab060957ac1ca8bf [id=f47382441b814a62b5ff875ae632b915]) successfully deleted
I20260505 07:54:12.311272 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 131) took 2703 ms. Trace:
W20260505 07:54:12.313202 1353 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.1:36755 (request call id 144) took 2209 ms (client timeout 1000 ms). Trace:
W20260505 07:54:12.313305 1353 rpcz_store.cc:269] 0505 07:54:10.104080 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.213969 (+109889us) service_pool.cc:224] Handling call
0505 07:54:12.313152 (+2099183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101,"reported_tablets":274}
I20260505 07:54:12.314606 2268 rpcz_store.cc:276] 0505 07:54:09.607786 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.607827 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:12.276878 (+2669051us) ts_tablet_manager.cc:1151] Deleting tablet 727cacd242844455915bd9963d2db19f
0505 07:54:12.276894 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.281829 (+ 4935us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.307336 (+ 25507us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.311255 (+ 3919us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2668853}
I20260505 07:54:12.315904 2268 tablet_replica.cc:333] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:12.316358 2268 raft_consensus.cc:2243] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:12.317188 1176 tablet_bootstrap.cc:492] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.317097 2268 raft_consensus.cc:2272] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.317307 1176 ts_tablet_manager.cc:1403] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.092s user 0.001s sys 0.008s
I20260505 07:54:12.318028 1176 raft_consensus.cc:359] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.318207 1176 raft_consensus.cc:385] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.318274 1176 raft_consensus.cc:740] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.318383 1172 tablet_bootstrap.cc:492] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:12.318548 1172 ts_tablet_manager.cc:1403] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.002s
I20260505 07:54:12.318485 1176 consensus_queue.cc:260] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.318852 2268 ts_tablet_manager.cc:1916] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.319103 1176 ts_tablet_manager.cc:1434] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:12.319360 1176 tablet_bootstrap.cc:492] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.321123 1176 tablet_bootstrap.cc:654] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.321035 1172 raft_consensus.cc:359] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.321235 1172 raft_consensus.cc:385] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.321305 1172 raft_consensus.cc:740] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.321514 1172 consensus_queue.cc:260] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.322042 1172 ts_tablet_manager.cc:1434] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:12.322283 1172 tablet_bootstrap.cc:492] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:12.325062 1172 tablet_bootstrap.cc:654] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.326660 907 raft_consensus.cc:2468] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:12.327800 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 727cacd242844455915bd9963d2db19f (table default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]) successfully deleted
I20260505 07:54:12.327849 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 147) took 2873 ms. Trace:
I20260505 07:54:12.327929 2447 rpcz_store.cc:276] 0505 07:54:09.454683 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.454729 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:12.327719 (+2872990us) ts_tablet_manager.cc:1151] Deleting tablet d896e290167b4d0081e8e8b831c8ae2a
0505 07:54:12.327731 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.327831 (+ 100us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2872841}
I20260505 07:54:12.328341 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 148) took 2873 ms. Trace:
W20260505 07:54:12.328416 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet d896e290167b4d0081e8e8b831c8ae2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d896e290167b4d0081e8e8b831c8ae2a already in progress: creating tablet
I20260505 07:54:12.328485 2447 rpcz_store.cc:276] 0505 07:54:09.455036 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.455081 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:12.328256 (+2873175us) ts_tablet_manager.cc:1151] Deleting tablet 4ba2cc218300472cb5fd6dc8cfb7811c
0505 07:54:12.328265 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.328326 (+ 61us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2873040}
I20260505 07:54:12.328917 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 149) took 2873 ms. Trace:
W20260505 07:54:12.328969 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 4ba2cc218300472cb5fd6dc8cfb7811c because the tablet was not found. No further retry: Not found: Tablet not found: 4ba2cc218300472cb5fd6dc8cfb7811c
I20260505 07:54:12.329341 2447 rpcz_store.cc:276] 0505 07:54:09.455329 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.455373 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:12.328816 (+2873443us) ts_tablet_manager.cc:1151] Deleting tablet e8bf49b787ef4de095448751464708a4
0505 07:54:12.328826 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.328901 (+ 75us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2873317}
I20260505 07:54:12.329741 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 150) took 2874 ms. Trace:
W20260505 07:54:12.330473 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e8bf49b787ef4de095448751464708a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8bf49b787ef4de095448751464708a4 already in progress: creating tablet
I20260505 07:54:12.330643 2447 rpcz_store.cc:276] 0505 07:54:09.481461 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.481522 (+ 61us) service_pool.cc:224] Handling call
0505 07:54:12.329650 (+2848128us) ts_tablet_manager.cc:1151] Deleting tablet a7d8265c05b74b28959d0de41fad1fde
0505 07:54:12.329658 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.329727 (+ 69us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2847939}
I20260505 07:54:12.330981 2447 tablet_replica.cc:333] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
W20260505 07:54:12.331132 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet a7d8265c05b74b28959d0de41fad1fde because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a7d8265c05b74b28959d0de41fad1fde already in progress: creating tablet
I20260505 07:54:12.331215 2447 raft_consensus.cc:2243] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.332312 2447 raft_consensus.cc:2272] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.332883 2447 ts_tablet_manager.cc:1916] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.336453 1176 tablet_bootstrap.cc:492] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.336607 1176 ts_tablet_manager.cc:1403] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.000s
I20260505 07:54:12.337257 1176 raft_consensus.cc:359] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.337517 1176 raft_consensus.cc:385] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.337603 1176 raft_consensus.cc:740] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.337839 1176 consensus_queue.cc:260] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.338483 1176 ts_tablet_manager.cc:1434] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:12.338792 1176 tablet_bootstrap.cc:492] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.343137 1176 tablet_bootstrap.cc:654] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.346517 781 raft_consensus.cc:3060] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:12.347693 991 consensus_peers.cc:597] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43e4c92b7c3140fdb2d6a290aaf0ad55. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:12.354027 2302 raft_consensus.cc:493] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.354246 2302 raft_consensus.cc:515] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.356981 2302 leader_election.cc:290] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.357263 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84f464e83cb412eade9978ea99f6685" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.357245 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84f464e83cb412eade9978ea99f6685" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.357712 907 raft_consensus.cc:2468] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:12.358004 988 leader_election.cc:343] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.358521 991 leader_election.cc:304] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.359002 2302 raft_consensus.cc:2804] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.359125 2302 raft_consensus.cc:493] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.359254 2302 raft_consensus.cc:3060] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.360127 2276 ts_tablet_manager.cc:1929] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.360240 2276 log.cc:1199] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/003cd4472b424aacbba8c5f89900e228
I20260505 07:54:12.360625 2276 ts_tablet_manager.cc:1950] T 003cd4472b424aacbba8c5f89900e228 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:12.362179 2302 raft_consensus.cc:515] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.363375 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84f464e83cb412eade9978ea99f6685" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:12.363955 988 leader_election.cc:343] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.369238 2302 leader_election.cc:290] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.369489 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84f464e83cb412eade9978ea99f6685" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:12.369884 907 raft_consensus.cc:3060] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.372448 1176 tablet_bootstrap.cc:492] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.372588 1176 ts_tablet_manager.cc:1403] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.000s
I20260505 07:54:12.373431 1176 raft_consensus.cc:359] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.373647 1176 raft_consensus.cc:385] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.374923 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 77) took 4708 ms. Trace:
I20260505 07:54:12.375028 2276 rpcz_store.cc:276] 0505 07:54:07.666366 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.666570 (+ 204us) service_pool.cc:224] Handling call
0505 07:54:12.041682 (+4375112us) ts_tablet_manager.cc:1151] Deleting tablet 003cd4472b424aacbba8c5f89900e228
0505 07:54:12.041695 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.267966 (+226271us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.360070 (+ 92104us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.374901 (+ 14831us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4374992}
I20260505 07:54:12.375452 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 78) took 4691 ms. Trace:
I20260505 07:54:12.375546 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 003cd4472b424aacbba8c5f89900e228 (table default.Table_bd283510c1f14a9fb29a9870d33c9971 [id=ae40934b942a4e0e8ac882d9dab6686f]) successfully deleted
I20260505 07:54:12.375537 2276 rpcz_store.cc:276] 0505 07:54:07.683622 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.683663 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:12.375322 (+4691659us) ts_tablet_manager.cc:1151] Deleting tablet 8eb7eaff8837485a8ae3ccb44c60471d
0505 07:54:12.375333 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.375435 (+ 102us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4691486}
I20260505 07:54:12.375808 2276 tablet_replica.cc:333] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:12.375998 2276 raft_consensus.cc:2243] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
W20260505 07:54:12.376154 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 8eb7eaff8837485a8ae3ccb44c60471d because the tablet was not found. No further retry: Not found: Tablet not found: 8eb7eaff8837485a8ae3ccb44c60471d
I20260505 07:54:12.376298 2276 raft_consensus.cc:2272] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.376833 2276 ts_tablet_manager.cc:1916] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.378643 1181 tablet_bootstrap.cc:492] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.379220 1351 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 162) took 1257 ms. Trace:
I20260505 07:54:12.379627 1351 rpcz_store.cc:276] 0505 07:54:11.121744 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.070014 (+948270us) service_pool.cc:224] Handling call
0505 07:54:12.243568 (+173554us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:12.244092 (+ 524us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:12.363377 (+119285us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:12.379053 (+ 15676us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:12.379163 (+ 110us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.246435 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.246513 (+ 78us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.246516 (+ 3us) write_op.cc:435] Acquired schema lock
0505 07:54:12.246519 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:12.246595 (+ 76us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.246606 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.246613 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.246698 (+ 85us) tablet.cc:701] Row locks acquired
0505 07:54:12.246702 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:12.246729 (+ 27us) write_op.cc:270] Start()
0505 07:54:12.246755 (+ 26us) write_op.cc:276] Timestamp: P: 1777967652246663 usec, L: 0
0505 07:54:12.246758 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.246880 (+ 122us) log.cc:844] Serialized 433 byte log entry
0505 07:54:12.361174 (+114294us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.361913 (+ 739us) write_op.cc:301] APPLY: starting
0505 07:54:12.361928 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.361966 (+ 38us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.362029 (+ 63us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.362123 (+ 94us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.362161 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:12.362172 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:12.362246 (+ 74us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.362296 (+ 50us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.362337 (+ 41us) write_op.cc:454] Released schema lock
0505 07:54:12.362342 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":699,"apply.run_cpu_time_us":487,"apply.run_wall_time_us":487,"num_ops":2,"prepare.queue_time_us":2002,"prepare.run_cpu_time_us":524,"prepare.run_wall_time_us":524,"replication_time_us":114399,"threads_started":1}]]}
I20260505 07:54:12.380200 1176 raft_consensus.cc:740] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.380432 1176 consensus_queue.cc:260] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.379504 1357 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 160) took 1293 ms. Trace:
I20260505 07:54:12.381022 1176 ts_tablet_manager.cc:1434] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260505 07:54:12.381309 1176 tablet_bootstrap.cc:492] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.380677 1357 rpcz_store.cc:276] 0505 07:54:11.086062 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.060696 (+974634us) service_pool.cc:224] Handling call
0505 07:54:12.242339 (+181643us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:12.242917 (+ 578us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:12.363566 (+120649us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:12.379367 (+ 15801us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:12.379447 (+ 80us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.245841 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.245932 (+ 91us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.245938 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:12.245940 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:12.246020 (+ 80us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.246029 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.246033 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.246103 (+ 70us) tablet.cc:701] Row locks acquired
0505 07:54:12.246106 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:12.246137 (+ 31us) write_op.cc:270] Start()
0505 07:54:12.246166 (+ 29us) write_op.cc:276] Timestamp: P: 1777967652246071 usec, L: 0
0505 07:54:12.246170 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.246338 (+ 168us) log.cc:844] Serialized 433 byte log entry
0505 07:54:12.361092 (+114754us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.361182 (+ 90us) write_op.cc:301] APPLY: starting
0505 07:54:12.361198 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.361247 (+ 49us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.361284 (+ 37us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.361430 (+ 146us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.361481 (+ 51us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:12.361494 (+ 13us) write_op.cc:312] APPLY: finished
0505 07:54:12.361594 (+ 100us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.361670 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.361712 (+ 42us) write_op.cc:454] Released schema lock
0505 07:54:12.361721 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.361818 (+ 97us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.361823 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":30,"apply.run_cpu_time_us":617,"apply.run_wall_time_us":617,"num_ops":2,"prepare.queue_time_us":2296,"prepare.run_cpu_time_us":566,"prepare.run_wall_time_us":566,"replication_time_us":114886}]]}
I20260505 07:54:12.383008 1176 tablet_bootstrap.cc:654] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.383891 1357 master_service.cc:954] ReplaceTablet: received request to replace tablet 472441bd29704be59aabb0f7a4964bb0 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:12.390648 2447 ts_tablet_manager.cc:1929] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:12.391299 2447 log.cc:1199] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/3d83e9d4243e409794ffe8a342bce6b3
I20260505 07:54:12.391916 2447 ts_tablet_manager.cc:1950] T 3d83e9d4243e409794ffe8a342bce6b3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:12.378940 1181 ts_tablet_manager.cc:1403] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.074s user 0.001s sys 0.002s
I20260505 07:54:12.392844 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 151) took 2910 ms. Trace:
I20260505 07:54:12.392976 2447 rpcz_store.cc:276] 0505 07:54:09.481934 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.481985 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:12.330945 (+2848960us) ts_tablet_manager.cc:1151] Deleting tablet 3d83e9d4243e409794ffe8a342bce6b3
0505 07:54:12.330953 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.370218 (+ 39265us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.390546 (+ 20328us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.392807 (+ 2261us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2848805}
I20260505 07:54:12.393388 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 3d83e9d4243e409794ffe8a342bce6b3 (table default.Table_563025047e4142d4807295cff836ea99 [id=5a7033e8ecc940e49794281e421153ea]) successfully deleted
I20260505 07:54:12.394039 2447 tablet_replica.cc:333] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:12.394297 2447 raft_consensus.cc:2243] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:12.394383 1357 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 163) took 1265 ms. Trace:
I20260505 07:54:12.394556 1357 rpcz_store.cc:276] 0505 07:54:11.129326 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.383792 (+1254466us) service_pool.cc:224] Handling call
0505 07:54:12.394122 (+ 10330us) spinlock_profiling.cc:243] Waited 9.92 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639e385b 00007fd65a477107 00007fd65a476e35 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:12.394345 (+ 223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22807296}
I20260505 07:54:12.394680 2447 raft_consensus.cc:2272] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.395021 1181 raft_consensus.cc:359] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:12.395134 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_ff4ec4dc16ec486d939b9b6e233ceaad"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:12.395272 2447 ts_tablet_manager.cc:1916] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:12.395905 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ff4ec4dc16ec486d939b9b6e233ceaad in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:12.397862 1353 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_f39ab9af40ff46588e5ea492e2e094c0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:12.398416 921 raft_consensus.cc:1275] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Refusing update from remote peer eac7e1e8caba43cdb4df80cbb80e891c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260505 07:54:12.399060 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet f840d5f58161468385d5c4f116b78c3d
I20260505 07:54:12.399549 2674 consensus_queue.cc:1048] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260505 07:54:12.399788 757 tablet_service.cc:1558] Processing DeleteTablet for tablet f840d5f58161468385d5c4f116b78c3d with delete_type TABLET_DATA_DELETED (replaced by tablet 06d87dc359cb48c2b5a6f5008f33427e) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.399889 887 tablet_service.cc:1558] Processing DeleteTablet for tablet f840d5f58161468385d5c4f116b78c3d with delete_type TABLET_DATA_DELETED (replaced by tablet 06d87dc359cb48c2b5a6f5008f33427e) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:12.411127 1181 raft_consensus.cc:385] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.411271 1181 raft_consensus.cc:740] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.411516 1181 consensus_queue.cc:260] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:12.412374 1350 catalog_manager.cc:6741] ReplaceTablet: tablet f840d5f58161468385d5c4f116b78c3d deleted and replaced by tablet 06d87dc359cb48c2b5a6f5008f33427e
I20260505 07:54:12.412539 1350 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 159) took 1381 ms. Trace:
I20260505 07:54:12.412670 1350 rpcz_store.cc:276] 0505 07:54:11.030623 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.059386 (+1028763us) service_pool.cc:224] Handling call
0505 07:54:12.412503 (+353117us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.342900 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.343054 (+ 154us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.343062 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:12.343066 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:12.343210 (+ 144us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.343224 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.343232 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.343331 (+ 99us) tablet.cc:701] Row locks acquired
0505 07:54:12.343335 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:12.348501 (+ 5166us) write_op.cc:270] Start()
0505 07:54:12.348555 (+ 54us) write_op.cc:276] Timestamp: P: 1777967652343293 usec, L: 0
0505 07:54:12.348561 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.348800 (+ 239us) log.cc:844] Serialized 622 byte log entry
0505 07:54:12.388430 (+ 39630us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.389194 (+ 764us) write_op.cc:301] APPLY: starting
0505 07:54:12.389203 (+ 9us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.389235 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.389276 (+ 41us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.389571 (+ 295us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.389625 (+ 54us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:12.389634 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:12.389688 (+ 54us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.389755 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.389770 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:12.389775 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.399676 (+ 9901us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.399681 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":622,"apply.run_cpu_time_us":544,"apply.run_wall_time_us":10450,"num_ops":2,"prepare.queue_time_us":3451,"prepare.run_cpu_time_us":908,"prepare.run_wall_time_us":5986,"replication_time_us":39853,"thread_start_us":5412,"threads_started":1}]]}
I20260505 07:54:12.412581 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet f840d5f58161468385d5c4f116b78c3d with delete_type TABLET_DATA_DELETED (replaced by tablet 06d87dc359cb48c2b5a6f5008f33427e) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:12.413523 1350 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_1f316191e5124266935010a5cdcc7a18" } modify_external_catalogs: true
I20260505 07:54:12.413868 1181 ts_tablet_manager.cc:1434] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.021s user 0.002s sys 0.000s
I20260505 07:54:12.413704 1350 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_1f316191e5124266935010a5cdcc7a18" } modify_external_catalogs: true
I20260505 07:54:12.416877 1357 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 165) took 1211 ms. Trace:
I20260505 07:54:12.417055 1357 rpcz_store.cc:276] 0505 07:54:11.205491 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.394964 (+1189473us) service_pool.cc:224] Handling call
0505 07:54:12.396015 (+ 1051us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:12.396501 (+ 486us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:12.416578 (+ 20077us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:12.416725 (+ 147us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:12.416827 (+ 102us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.397040 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.397139 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.397144 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:12.397146 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:12.397230 (+ 84us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.397241 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.397245 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.397312 (+ 67us) tablet.cc:701] Row locks acquired
0505 07:54:12.397314 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:12.397346 (+ 32us) write_op.cc:270] Start()
0505 07:54:12.397371 (+ 25us) write_op.cc:276] Timestamp: P: 1777967652397265 usec, L: 0
0505 07:54:12.397374 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.397534 (+ 160us) log.cc:844] Serialized 433 byte log entry
0505 07:54:12.415955 (+ 18421us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.416054 (+ 99us) write_op.cc:301] APPLY: starting
0505 07:54:12.416067 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.416114 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.416147 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.416264 (+ 117us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.416314 (+ 50us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:12.416326 (+ 12us) write_op.cc:312] APPLY: finished
0505 07:54:12.416412 (+ 86us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.416483 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.416498 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:12.416503 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":55,"num_ops":2,"prepare.queue_time_us":46,"prepare.run_cpu_time_us":555,"prepare.run_wall_time_us":555,"replication_time_us":18546}]]}
I20260505 07:54:12.418042 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_ab051f01f8054d1b91cbe477cef39adc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:12.418733 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ab051f01f8054d1b91cbe477cef39adc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:12.419695 1353 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f39ab9af40ff46588e5ea492e2e094c0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:12.430593 1172 tablet_bootstrap.cc:492] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:12.431125 1172 ts_tablet_manager.cc:1403] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.109s user 0.009s sys 0.015s
I20260505 07:54:12.431639 907 raft_consensus.cc:2468] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:12.434592 991 leader_election.cc:304] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.435477 2302 raft_consensus.cc:2804] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:12.435644 2302 raft_consensus.cc:697] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:12.435917 2302 consensus_queue.cc:237] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.441587 2564 raft_consensus.cc:493] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.441731 2564 raft_consensus.cc:515] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.442389 2564 leader_election.cc:290] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.443015 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bacc97d7534284970f218cf68ac75b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:12.444097 856 leader_election.cc:343] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.444931 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bacc97d7534284970f218cf68ac75b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:12.445672 854 leader_election.cc:343] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.449357 2405 raft_consensus.cc:493] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.449504 2405 raft_consensus.cc:515] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.450074 2405 leader_election.cc:290] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.451360 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1175f7deec14fe39098ec89ab3bc049" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.451671 781 raft_consensus.cc:2468] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:12.452097 854 leader_election.cc:304] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.452888 988 leader_election.cc:304] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:12.453233 2564 raft_consensus.cc:2749] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.453254 2307 raft_consensus.cc:2804] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.453331 2307 raft_consensus.cc:493] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.453390 2307 raft_consensus.cc:3060] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.453889 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1175f7deec14fe39098ec89ab3bc049" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.457284 2307 raft_consensus.cc:515] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.457835 2307 leader_election.cc:290] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.458554 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1175f7deec14fe39098ec89ab3bc049" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:12.458760 781 raft_consensus.cc:3060] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.458874 1181 tablet_bootstrap.cc:492] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.461429 781 raft_consensus.cc:2468] T a1175f7deec14fe39098ec89ab3bc049 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:12.461954 988 leader_election.cc:304] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:12.463976 2307 raft_consensus.cc:2804] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:12.464144 2307 raft_consensus.cc:697] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:12.464426 2307 consensus_queue.cc:237] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.464570 2268 ts_tablet_manager.cc:1929] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.467015 2268 log.cc:1199] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/02412333633a4cdebe95a144fa6c363c
I20260505 07:54:12.469048 2268 ts_tablet_manager.cc:1950] T 02412333633a4cdebe95a144fa6c363c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.473322 1176 tablet_bootstrap.cc:492] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.473685 1176 ts_tablet_manager.cc:1403] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.092s user 0.004s sys 0.002s
I20260505 07:54:12.474841 1176 raft_consensus.cc:359] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.479511 1181 tablet_bootstrap.cc:654] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.484463 1176 raft_consensus.cc:385] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.484859 1176 raft_consensus.cc:740] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.485371 1176 consensus_queue.cc:260] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:12.486874 1317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.62:33621, user_credentials={real_user=slave}} blocked reactor thread for 65129.6us
I20260505 07:54:12.474812 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 132) took 2866 ms. Trace:
I20260505 07:54:12.488315 2268 rpcz_store.cc:276] 0505 07:54:09.608281 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.608334 (+ 53us) service_pool.cc:224] Handling call
0505 07:54:12.315276 (+2706942us) ts_tablet_manager.cc:1151] Deleting tablet 02412333633a4cdebe95a144fa6c363c
0505 07:54:12.315787 (+ 511us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.386665 (+ 70878us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.464519 (+ 77854us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.474622 (+ 10103us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2706752}
I20260505 07:54:12.488554 2276 ts_tablet_manager.cc:1929] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.488507 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1175f7deec14fe39098ec89ab3bc049" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:12.488610 2564 raft_consensus.cc:493] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.488698 2276 log.cc:1199] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/ff7ed926a3ff4e7993864de734ca3c50
I20260505 07:54:12.488793 2564 raft_consensus.cc:515] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.489063 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 02412333633a4cdebe95a144fa6c363c (table default.Table_0654ae2daf60435697eeca15ba0c19d7 [id=13d54930c52f4e9eae01bc80a3f1d565]) successfully deleted
I20260505 07:54:12.489173 2276 ts_tablet_manager.cc:1950] T ff7ed926a3ff4e7993864de734ca3c50 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:12.489130 1172 raft_consensus.cc:359] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.489463 2564 leader_election.cc:290] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.489667 1172 raft_consensus.cc:385] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.490088 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 79) took 4801 ms. Trace:
I20260505 07:54:12.490609 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9963a0f1a54ee9a73486bb1801ee74" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.490628 2276 rpcz_store.cc:276] 0505 07:54:07.688698 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.688742 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:12.375765 (+4687023us) ts_tablet_manager.cc:1151] Deleting tablet ff7ed926a3ff4e7993864de734ca3c50
0505 07:54:12.375774 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.429952 (+ 54178us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.459764 (+ 29812us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.490051 (+ 30287us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4686866}
I20260505 07:54:12.490718 1172 raft_consensus.cc:740] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.491109 2276 tablet_replica.cc:333] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:12.491350 2276 raft_consensus.cc:2243] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.491371 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet ff7ed926a3ff4e7993864de734ca3c50 (table default.Table_1b644b185b194940a8a8a90595ae9e90 [id=8e18fbfbd36340fabddf08dc99fea437]) successfully deleted
I20260505 07:54:12.491531 2276 raft_consensus.cc:2272] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:12.491492 854 leader_election.cc:343] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.491655 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 133) took 2879 ms. Trace:
I20260505 07:54:12.491742 2268 rpcz_store.cc:276] 0505 07:54:09.612611 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.612653 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:12.489632 (+2876979us) ts_tablet_manager.cc:1151] Deleting tablet 3075e87e078f4d2aa1d329d18902354b
0505 07:54:12.489646 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.491633 (+ 1987us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2876790}
I20260505 07:54:12.491890 907 raft_consensus.cc:2468] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:12.492246 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 3075e87e078f4d2aa1d329d18902354b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3075e87e078f4d2aa1d329d18902354b already in progress: creating tablet
I20260505 07:54:12.491915 1172 consensus_queue.cc:260] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.492481 2276 ts_tablet_manager.cc:1916] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.492501 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9963a0f1a54ee9a73486bb1801ee74" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:12.493170 2268 tablet_replica.cc:333] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:12.493330 1172 ts_tablet_manager.cc:1434] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.062s user 0.005s sys 0.000s
W20260505 07:54:12.493407 856 leader_election.cc:343] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.493567 856 leader_election.cc:304] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.493659 2268 raft_consensus.cc:2243] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:12.493994 2268 raft_consensus.cc:2272] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.494133 2505 raft_consensus.cc:2749] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.494580 2268 ts_tablet_manager.cc:1916] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.505932 1176 ts_tablet_manager.cc:1434] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.032s user 0.001s sys 0.001s
I20260505 07:54:12.509958 1181 tablet_bootstrap.cc:492] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.510172 1181 ts_tablet_manager.cc:1403] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.051s user 0.004s sys 0.002s
I20260505 07:54:12.511092 1181 raft_consensus.cc:359] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.511327 1181 raft_consensus.cc:385] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.511420 1181 raft_consensus.cc:740] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.511705 1181 consensus_queue.cc:260] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.512255 1181 ts_tablet_manager.cc:1434] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:12.512959 1181 tablet_bootstrap.cc:492] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.514667 1181 tablet_bootstrap.cc:654] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.516602 1176 tablet_bootstrap.cc:492] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.518282 1176 tablet_bootstrap.cc:654] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.498915 2276 ts_tablet_manager.cc:1929] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.501313 1172 tablet_bootstrap.cc:492] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:12.519729 2276 log.cc:1199] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/f121fdd001ec4ea38ccd24a6afd9e51c
I20260505 07:54:12.520176 2276 ts_tablet_manager.cc:1950] T f121fdd001ec4ea38ccd24a6afd9e51c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:12.520529 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 80) took 4823 ms. Trace:
I20260505 07:54:12.520640 2276 rpcz_store.cc:276] 0505 07:54:07.696815 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.696862 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:12.491050 (+4794188us) ts_tablet_manager.cc:1151] Deleting tablet f121fdd001ec4ea38ccd24a6afd9e51c
0505 07:54:12.491059 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.495852 (+ 4793us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.498872 (+ 3020us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.520465 (+ 21593us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4794040}
I20260505 07:54:12.520994 2276 tablet_replica.cc:333] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:12.521154 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet f121fdd001ec4ea38ccd24a6afd9e51c (table default.Table_3a76cf09eaed4c9a8c267714a164dd66 [id=6be00fd9040d4febaef6b16dec369a3a]) successfully deleted
I20260505 07:54:12.523232 1172 tablet_bootstrap.cc:654] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.523780 2268 ts_tablet_manager.cc:1929] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.523897 2268 log.cc:1199] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/e29ba55a846548679cb3b86305d0d6ed
I20260505 07:54:12.524256 2268 ts_tablet_manager.cc:1950] T e29ba55a846548679cb3b86305d0d6ed P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.525025 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 134) took 2911 ms. Trace:
I20260505 07:54:12.525126 2268 rpcz_store.cc:276] 0505 07:54:09.613649 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.613687 (+ 38us) service_pool.cc:224] Handling call
0505 07:54:12.493117 (+2879430us) ts_tablet_manager.cc:1151] Deleting tablet e29ba55a846548679cb3b86305d0d6ed
0505 07:54:12.493128 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.520932 (+ 27804us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.523732 (+ 2800us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.525006 (+ 1274us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2879255}
I20260505 07:54:12.525573 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet e29ba55a846548679cb3b86305d0d6ed (table default.Table_fd75ee1f117344169ccafd58dc18408f [id=5cd6a3d7d65d4a43943fa891ca480d7e]) successfully deleted
I20260505 07:54:12.527649 2276 raft_consensus.cc:2243] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.528103 2276 raft_consensus.cc:2272] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.528642 2276 ts_tablet_manager.cc:1916] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.528920 2302 raft_consensus.cc:493] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.529089 2302 raft_consensus.cc:515] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.530117 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db162dfad2448e8ac428b3d3fbe17f7" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.530323 781 raft_consensus.cc:2468] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:12.530793 988 leader_election.cc:304] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:12.531054 2405 raft_consensus.cc:2804] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.531126 2405 raft_consensus.cc:493] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.531190 2405 raft_consensus.cc:3060] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.534164 2405 raft_consensus.cc:515] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.534472 2276 ts_tablet_manager.cc:1929] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.534611 2276 log.cc:1199] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/4783dd0b2fbe40219852d850fc0d9a82
I20260505 07:54:12.535748 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0db162dfad2448e8ac428b3d3fbe17f7" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:12.535948 781 raft_consensus.cc:3060] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.535979 2405 leader_election.cc:290] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:12.536521 991 leader_election.cc:343] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 0db162dfad2448e8ac428b3d3fbe17f7
I20260505 07:54:12.537163 2302 leader_election.cc:290] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:12.537650 991 leader_election.cc:343] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 0db162dfad2448e8ac428b3d3fbe17f7
I20260505 07:54:12.539012 781 raft_consensus.cc:2468] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:12.539489 988 leader_election.cc:304] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:12.540529 2405 raft_consensus.cc:2804] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:12.540606 1181 tablet_bootstrap.cc:492] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.540807 1181 ts_tablet_manager.cc:1403] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.001s
I20260505 07:54:12.541677 1181 raft_consensus.cc:359] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.541917 1181 raft_consensus.cc:385] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.542037 1181 raft_consensus.cc:740] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.540827 2405 raft_consensus.cc:697] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:12.542438 2405 consensus_queue.cc:237] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.543465 1181 consensus_queue.cc:260] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.544266 1181 ts_tablet_manager.cc:1434] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:12.544613 1181 tablet_bootstrap.cc:492] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.546341 1181 tablet_bootstrap.cc:654] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.547899 1172 tablet_bootstrap.cc:492] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:12.546831 2268 tablet_replica.cc:333] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:12.548280 1172 ts_tablet_manager.cc:1403] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.048s user 0.008s sys 0.003s
I20260505 07:54:12.548437 2268 raft_consensus.cc:2243] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:12.548484 2276 ts_tablet_manager.cc:1950] T 4783dd0b2fbe40219852d850fc0d9a82 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:12.548704 2447 ts_tablet_manager.cc:1929] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.548897 2268 raft_consensus.cc:2272] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.548933 2447 log.cc:1199] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/7585f00119e14006990ed551667b4486
I20260505 07:54:12.549508 2268 ts_tablet_manager.cc:1916] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.549553 2447 ts_tablet_manager.cc:1950] T 7585f00119e14006990ed551667b4486 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:12.550104 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 152) took 3067 ms. Trace:
I20260505 07:54:12.550275 2447 rpcz_store.cc:276] 0505 07:54:09.482246 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.482282 (+ 36us) service_pool.cc:224] Handling call
0505 07:54:12.393958 (+2911676us) ts_tablet_manager.cc:1151] Deleting tablet 7585f00119e14006990ed551667b4486
0505 07:54:12.393970 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.476357 (+ 82387us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.548627 (+ 72270us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.550051 (+ 1424us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2911548}
I20260505 07:54:12.550864 2447 tablet_replica.cc:333] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:12.551203 2447 raft_consensus.cc:2243] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.550937 1172 raft_consensus.cc:359] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.551477 2447 raft_consensus.cc:2272] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.551823 1172 raft_consensus.cc:385] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.552130 1172 raft_consensus.cc:740] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.552340 2447 ts_tablet_manager.cc:1916] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.552567 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 7585f00119e14006990ed551667b4486 (table default.Table_c61fca437ddb413e934595a23b52f40f [id=a14990bc69324c54a3c7e2b7d6116d11]) successfully deleted
I20260505 07:54:12.552528 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:12.553409 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:12.553783 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:12.554003 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:12.553412 1172 consensus_queue.cc:260] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.554237 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:12.554431 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:12.554591 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:12.554770 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:12.555011 2302 raft_consensus.cc:493] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.555090 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:12.555254 2302 raft_consensus.cc:515] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.555483 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:12.557206 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66082a7a75e345639e7a268727922fbb" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:12.557818 988 leader_election.cc:343] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.558645 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66082a7a75e345639e7a268727922fbb" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.558911 920 raft_consensus.cc:2468] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:12.559322 991 leader_election.cc:304] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.559563 2307 raft_consensus.cc:2804] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.559684 2307 raft_consensus.cc:493] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.559755 2307 raft_consensus.cc:3060] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.560180 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 81) took 4858 ms. Trace:
I20260505 07:54:12.560302 2276 rpcz_store.cc:276] 0505 07:54:07.701557 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.701604 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:12.520902 (+4819298us) ts_tablet_manager.cc:1151] Deleting tablet 4783dd0b2fbe40219852d850fc0d9a82
0505 07:54:12.520910 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.527973 (+ 7063us) spinlock_profiling.cc:243] Waited 6.42 ms on lock 0x6160000c2e54. stack: 00007f1644039086 00007f1644038e40 00007f1642ff23ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007f164ff7c7e0 00007f16512d0283 00007f1653e95a77 00007f1653ec8b44 00007f1653ea9560 00007f1653ea92ed 0000000000aa4cac 00007f164407ef6f 00007f164408228b 00007f1644082108
0505 07:54:12.531625 (+ 3652us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.534391 (+ 2766us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.560145 (+ 25754us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14761728,"tablet-delete.queue_time_us":4819142}
I20260505 07:54:12.562099 2276 tablet_replica.cc:333] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:12.562307 2276 raft_consensus.cc:2243] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:12.562609 2276 pending_rounds.cc:70] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:54:12.562687 2276 pending_rounds.cc:77] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7282555496686583808 op_type: NO_OP noop_request { }
I20260505 07:54:12.562796 2276 raft_consensus.cc:2889] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:12.562700 2307 raft_consensus.cc:515] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.562901 2276 raft_consensus.cc:2272] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.563388 2276 ts_tablet_manager.cc:1916] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.564194 1181 tablet_bootstrap.cc:492] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.564316 1181 ts_tablet_manager.cc:1403] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.003s
I20260505 07:54:12.565035 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66082a7a75e345639e7a268727922fbb" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:12.565168 1181 raft_consensus.cc:359] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.565270 920 raft_consensus.cc:3060] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.565325 1181 raft_consensus.cc:385] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.565387 1181 raft_consensus.cc:740] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.569520 2276 ts_tablet_manager.cc:1929] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:12.569635 2276 log.cc:1199] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/6da10815e0dc41a1a07b6786882fffd3
I20260505 07:54:12.569205 1172 ts_tablet_manager.cc:1434] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.021s user 0.003s sys 0.000s
I20260505 07:54:12.563856 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 4783dd0b2fbe40219852d850fc0d9a82 (table default.Table_0555c54d8e394cb1a8509950422c9ac8 [id=e59beab126684f548325a29eb663177b]) successfully deleted
I20260505 07:54:12.570036 2276 ts_tablet_manager.cc:1950] T 6da10815e0dc41a1a07b6786882fffd3 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:12.570092 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:12.570753 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 82) took 4858 ms. Trace:
I20260505 07:54:12.570871 2276 rpcz_store.cc:276] 0505 07:54:07.712328 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.713065 (+ 737us) service_pool.cc:224] Handling call
0505 07:54:12.562046 (+4848981us) ts_tablet_manager.cc:1151] Deleting tablet 6da10815e0dc41a1a07b6786882fffd3
0505 07:54:12.562054 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.566464 (+ 4410us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.569433 (+ 2969us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.570706 (+ 1273us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4848828}
I20260505 07:54:12.571201 2276 tablet_replica.cc:333] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:12.571412 2276 raft_consensus.cc:2243] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.571607 2276 raft_consensus.cc:2272] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.571748 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 6da10815e0dc41a1a07b6786882fffd3 (table default.Table_344c494090d44fa8b4102b525e7ca099 [id=d8050f3674fe49f7a8af288e6ea57a69]) successfully deleted
I20260505 07:54:12.573137 2307 leader_election.cc:290] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.573258 2276 ts_tablet_manager.cc:1916] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.573405 1172 tablet_bootstrap.cc:492] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:12.573396 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66082a7a75e345639e7a268727922fbb" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:12.574353 988 leader_election.cc:343] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.575960 2302 leader_election.cc:290] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.578258 920 raft_consensus.cc:2468] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:12.578895 991 leader_election.cc:304] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.579366 2302 raft_consensus.cc:2804] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:12.579558 1172 tablet_bootstrap.cc:654] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.585006 2302 raft_consensus.cc:697] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:12.585497 2302 consensus_queue.cc:237] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.587786 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:12.588073 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:12.588341 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:12.588563 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:12.592366 1181 consensus_queue.cc:260] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.595086 1181 ts_tablet_manager.cc:1434] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.031s user 0.001s sys 0.001s
I20260505 07:54:12.595854 1176 tablet_bootstrap.cc:492] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.596004 1176 ts_tablet_manager.cc:1403] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.080s user 0.002s sys 0.001s
I20260505 07:54:12.596807 1176 raft_consensus.cc:359] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.597070 1181 tablet_bootstrap.cc:492] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.596993 1176 raft_consensus.cc:385] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.597458 1176 raft_consensus.cc:740] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.597725 1176 consensus_queue.cc:260] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.598505 1176 ts_tablet_manager.cc:1434] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:12.599846 1355 catalog_manager.cc:5671] T 4cb7b3fded5d4def8db8d5c1e77ddcda P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:12.602828 1176 tablet_bootstrap.cc:492] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.603864 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:12.605202 1181 tablet_bootstrap.cc:654] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.605278 1176 tablet_bootstrap.cc:654] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.605559 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:12.615877 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:12.616285 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:12.616691 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
I20260505 07:54:12.616994 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:12.617928 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:12.618351 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:12.618829 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:12.619740 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:12.623261 2268 ts_tablet_manager.cc:1929] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.623409 2268 log.cc:1199] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/46274a44a06b4306b5dac43c66c85a80
I20260505 07:54:12.623849 2268 ts_tablet_manager.cc:1950] T 46274a44a06b4306b5dac43c66c85a80 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.623873 1181 tablet_bootstrap.cc:492] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.624081 1181 ts_tablet_manager.cc:1403] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.002s
I20260505 07:54:12.624527 2673 raft_consensus.cc:493] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.624683 2673 raft_consensus.cc:515] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:12.624950 1181 raft_consensus.cc:359] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.625098 1181 raft_consensus.cc:385] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.625144 1181 raft_consensus.cc:740] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.625314 1181 consensus_queue.cc:260] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.625778 2673 leader_election.cc:290] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.625794 1181 ts_tablet_manager.cc:1434] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:12.626072 1181 tablet_bootstrap.cc:492] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.626214 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe19306f7df49fdb771d21f6560e697" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.626394 920 raft_consensus.cc:2468] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:12.626371 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe19306f7df49fdb771d21f6560e697" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:12.626524 1047 raft_consensus.cc:2468] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:12.626796 718 leader_election.cc:304] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:12.627074 2673 raft_consensus.cc:2804] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.627148 2673 raft_consensus.cc:493] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.627209 2673 raft_consensus.cc:3060] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.627746 1181 tablet_bootstrap.cc:654] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.628545 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:12.628758 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:12.629065 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:12.629254 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:12.629595 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:12.630054 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:12.633379 2564 raft_consensus.cc:493] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:12.633510 2564 raft_consensus.cc:515] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.633746 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:12.634042 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:12.634090 2564 leader_election.cc:290] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.634878 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
W20260505 07:54:12.635509 856 leader_election.cc:343] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: aab3850799254dda9716fd5bf9404159
I20260505 07:54:12.642477 2280 consensus_queue.cc:1048] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:12.642645 2405 consensus_queue.cc:1048] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:12.643812 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:12.644245 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:12.645282 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:12.645677 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:12.645852 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 135) took 3031 ms. Trace:
I20260505 07:54:12.645934 2268 rpcz_store.cc:276] 0505 07:54:09.613985 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.614019 (+ 34us) service_pool.cc:224] Handling call
0505 07:54:12.546779 (+2932760us) ts_tablet_manager.cc:1151] Deleting tablet 46274a44a06b4306b5dac43c66c85a80
0505 07:54:12.546790 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.566646 (+ 19856us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.623198 (+ 56552us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.645830 (+ 22632us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2932603}
I20260505 07:54:12.646420 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 46274a44a06b4306b5dac43c66c85a80 (table default.Table_614ea89c84d342f0a1c4912e8e77fe47 [id=f1230f25280e45269ffbd95bbe8dbbfc]) successfully deleted
I20260505 07:54:12.646504 2268 tablet_replica.cc:333] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:12.646742 2268 raft_consensus.cc:2243] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
W20260505 07:54:12.647742 718 consensus_peers.cc:597] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3d6494590052459485059f5dce46f34f. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:12.647903 2268 raft_consensus.cc:2272] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:12.648445 854 leader_election.cc:343] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: aab3850799254dda9716fd5bf9404159
I20260505 07:54:12.648543 2268 ts_tablet_manager.cc:1916] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.648562 854 leader_election.cc:304] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.648947 2564 raft_consensus.cc:2749] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:12.649987 1181 tablet_bootstrap.cc:492] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.650118 1181 ts_tablet_manager.cc:1403] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.000s
I20260505 07:54:12.650875 1181 raft_consensus.cc:359] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.651026 1181 raft_consensus.cc:385] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.651070 1181 raft_consensus.cc:740] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.651129 2672 consensus_queue.cc:1048] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:12.651260 1181 consensus_queue.cc:260] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.651779 1181 ts_tablet_manager.cc:1434] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:12.651975 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:12.652343 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:12.652608 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:12.652807 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:12.653246 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 21480c42b73e4bf0ab2453b085a867c0 (table default.Table_925028e721e24ba68f31fe1040d4c2fd [id=0faa6676bab04919984b580299143271]). Expected version 1 got 0
I20260505 07:54:12.653505 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:12.653916 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:12.655411 1181 tablet_bootstrap.cc:492] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.656762 1878 consensus_queue.cc:1048] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:12.657644 1181 tablet_bootstrap.cc:654] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.659992 1176 tablet_bootstrap.cc:492] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.660131 1176 ts_tablet_manager.cc:1403] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.057s user 0.000s sys 0.004s
I20260505 07:54:12.660910 1176 raft_consensus.cc:359] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.661094 1176 raft_consensus.cc:385] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.661206 1176 raft_consensus.cc:740] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.661458 1176 consensus_queue.cc:260] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.662156 1176 ts_tablet_manager.cc:1434] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:12.662439 1176 tablet_bootstrap.cc:492] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.666965 1353 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 164) took 1462 ms. Trace:
I20260505 07:54:12.667081 1353 rpcz_store.cc:276] 0505 07:54:11.204919 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.384646 (+1179727us) service_pool.cc:224] Handling call
0505 07:54:12.397786 (+ 13140us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d6b13 00007fd65a46a7c6 00007fd65a46a4f5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:12.475384 (+ 77598us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:12.476332 (+ 948us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:12.666671 (+190339us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:12.666794 (+ 123us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:12.666909 (+ 115us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.484770 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.484896 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.484902 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:12.484905 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:12.485042 (+ 137us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.485056 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.485061 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.485144 (+ 83us) tablet.cc:701] Row locks acquired
0505 07:54:12.485147 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:12.485185 (+ 38us) write_op.cc:270] Start()
0505 07:54:12.485219 (+ 34us) write_op.cc:276] Timestamp: P: 1777967652485095 usec, L: 0
0505 07:54:12.485223 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.485422 (+ 199us) log.cc:844] Serialized 433 byte log entry
0505 07:54:12.664932 (+179510us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.665160 (+ 228us) write_op.cc:301] APPLY: starting
0505 07:54:12.665177 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.665224 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.665266 (+ 42us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.665389 (+ 123us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.665427 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:12.665434 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:12.665520 (+ 86us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.665588 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.665607 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:12.665615 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.665687 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.665694 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":30592640,"child_traces":[["op",{"apply.queue_time_us":175,"apply.run_cpu_time_us":520,"apply.run_wall_time_us":519,"num_ops":2,"prepare.queue_time_us":7922,"prepare.run_cpu_time_us":734,"prepare.run_wall_time_us":734,"replication_time_us":179675}]]}
I20260505 07:54:12.668005 1353 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_fe77d748257e4d848042a96b91636eb2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:12.668581 1353 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fe77d748257e4d848042a96b91636eb2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:12.667692 1181 tablet_bootstrap.cc:492] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.668916 1181 ts_tablet_manager.cc:1403] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260505 07:54:12.669937 1357 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 167) took 1457 ms. Trace:
I20260505 07:54:12.670055 1357 rpcz_store.cc:276] 0505 07:54:11.212249 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.417849 (+1205600us) service_pool.cc:224] Handling call
0505 07:54:12.475224 (+ 57375us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:12.476332 (+ 1108us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:12.669696 (+193364us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:12.669790 (+ 94us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:12.669890 (+ 100us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.485528 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.485641 (+ 113us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.485646 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:12.485648 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:12.485745 (+ 97us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.485754 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.485758 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.485814 (+ 56us) tablet.cc:701] Row locks acquired
0505 07:54:12.485816 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:12.485853 (+ 37us) write_op.cc:270] Start()
0505 07:54:12.485882 (+ 29us) write_op.cc:276] Timestamp: P: 1777967652485765 usec, L: 0
0505 07:54:12.485886 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.486034 (+ 148us) log.cc:844] Serialized 433 byte log entry
0505 07:54:12.665007 (+178973us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.665766 (+ 759us) write_op.cc:301] APPLY: starting
0505 07:54:12.665780 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.665814 (+ 34us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.665839 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.665935 (+ 96us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.665966 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:12.665976 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:12.666040 (+ 64us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.666084 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.666098 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:12.666104 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.666167 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.666174 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":725,"apply.run_cpu_time_us":395,"apply.run_wall_time_us":395,"mutex_wait_us":7844,"num_ops":2,"prepare.queue_time_us":8707,"prepare.run_cpu_time_us":578,"prepare.run_wall_time_us":578,"replication_time_us":179102}]]}
I20260505 07:54:12.670858 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_6e7a27e38f8a4feb82d93ea27356ba60"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:12.671363 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6e7a27e38f8a4feb82d93ea27356ba60 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:12.670558 1181 raft_consensus.cc:359] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.671865 1181 raft_consensus.cc:385] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.672013 1181 raft_consensus.cc:740] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.673254 1181 consensus_queue.cc:260] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.674140 1181 ts_tablet_manager.cc:1434] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:12.678349 1181 tablet_bootstrap.cc:492] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.679838 772 raft_consensus.cc:1275] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Refusing update from remote peer d63a3e535f8e4dd4a88e447253405b77: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260505 07:54:12.680500 2307 consensus_queue.cc:1048] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260505 07:54:12.682130 1181 tablet_bootstrap.cc:654] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.683900 2564 raft_consensus.cc:493] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.684037 2564 raft_consensus.cc:515] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.684705 2564 leader_election.cc:290] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.685313 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77ff90ec2ff4b6eb1795d149459c23a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:12.685801 854 leader_election.cc:343] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.685913 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77ff90ec2ff4b6eb1795d149459c23a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:12.686515 856 leader_election.cc:343] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.686647 856 leader_election.cc:304] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.687150 2564 raft_consensus.cc:2749] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.688452 1357 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 169) took 1452 ms. Trace:
I20260505 07:54:12.688596 1357 rpcz_store.cc:276] 0505 07:54:11.236161 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.670678 (+1434517us) service_pool.cc:224] Handling call
0505 07:54:12.671464 (+ 786us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:12.671935 (+ 471us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:12.688232 (+ 16297us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:12.688333 (+ 101us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:12.688414 (+ 81us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.672427 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.672560 (+ 133us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.672568 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:12.672571 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:12.672672 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.672681 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.672688 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.672765 (+ 77us) tablet.cc:701] Row locks acquired
0505 07:54:12.672769 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:12.672799 (+ 30us) write_op.cc:270] Start()
0505 07:54:12.672836 (+ 37us) write_op.cc:276] Timestamp: P: 1777967652672690 usec, L: 0
0505 07:54:12.672842 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.673029 (+ 187us) log.cc:844] Serialized 433 byte log entry
0505 07:54:12.686305 (+ 13276us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.686931 (+ 626us) write_op.cc:301] APPLY: starting
0505 07:54:12.686940 (+ 9us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.686969 (+ 29us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.686995 (+ 26us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.687105 (+ 110us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.687140 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:12.687150 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:12.687201 (+ 51us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.687238 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.687253 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:12.687258 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.687329 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.687332 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":597,"apply.run_cpu_time_us":375,"apply.run_wall_time_us":375,"num_ops":2,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":691,"prepare.run_wall_time_us":674,"replication_time_us":13449}]]}
I20260505 07:54:12.689863 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_3f9cf31daaba4d4f95774e2628b068cb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:12.690515 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f9cf31daaba4d4f95774e2628b068cb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:12.692780 1353 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 168) took 1465 ms. Trace:
I20260505 07:54:12.692888 1353 rpcz_store.cc:276] 0505 07:54:11.226975 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.667794 (+1440819us) service_pool.cc:224] Handling call
0505 07:54:12.668690 (+ 896us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:12.669188 (+ 498us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:12.692475 (+ 23287us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:12.692640 (+ 165us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:12.692726 (+ 86us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.669729 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.669844 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.669850 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:12.669853 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:12.669959 (+ 106us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.669974 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.669980 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.670061 (+ 81us) tablet.cc:701] Row locks acquired
0505 07:54:12.670064 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:12.670096 (+ 32us) write_op.cc:270] Start()
0505 07:54:12.670123 (+ 27us) write_op.cc:276] Timestamp: P: 1777967652669987 usec, L: 0
0505 07:54:12.670127 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.670276 (+ 149us) log.cc:844] Serialized 433 byte log entry
0505 07:54:12.686244 (+ 15968us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.686388 (+ 144us) write_op.cc:301] APPLY: starting
0505 07:54:12.686403 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.686437 (+ 34us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.686471 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.686599 (+ 128us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.686632 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:12.686639 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:12.686702 (+ 63us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.686758 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.686776 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:12.686781 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.686858 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.686861 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":101,"apply.run_cpu_time_us":447,"apply.run_wall_time_us":447,"num_ops":2,"prepare.queue_time_us":42,"prepare.run_cpu_time_us":626,"prepare.run_wall_time_us":626,"replication_time_us":16087}]]}
I20260505 07:54:12.698629 1181 tablet_bootstrap.cc:492] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.698753 1181 ts_tablet_manager.cc:1403] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.000s
I20260505 07:54:12.699487 1181 raft_consensus.cc:359] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.699668 1181 raft_consensus.cc:385] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.699712 1181 raft_consensus.cc:740] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.699865 1181 consensus_queue.cc:260] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.700477 1181 ts_tablet_manager.cc:1434] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:12.700778 1181 tablet_bootstrap.cc:492] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.701303 1176 tablet_bootstrap.cc:654] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.703023 1181 tablet_bootstrap.cc:654] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.709830 1353 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_101c994cc1964e8a88f0bdb4fc08b1b9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:12.710342 1353 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_101c994cc1964e8a88f0bdb4fc08b1b9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:12.711447 991 consensus_peers.cc:597] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4783dd0b2fbe40219852d850fc0d9a82. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:12.713665 2564 raft_consensus.cc:493] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.713792 2564 raft_consensus.cc:515] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.714357 2564 leader_election.cc:290] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.714329 2673 raft_consensus.cc:515] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:12.715045 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2045202714c14aa388d1e8bc18e2063d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.715399 1176 tablet_bootstrap.cc:492] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.715504 1176 ts_tablet_manager.cc:1403] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.053s user 0.003s sys 0.000s
I20260505 07:54:12.715474 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2045202714c14aa388d1e8bc18e2063d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:12.715517 854 leader_election.cc:343] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:12.716171 856 leader_election.cc:343] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.716269 856 leader_election.cc:304] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.716610 2564 raft_consensus.cc:2749] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.720781 1357 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 170) took 1479 ms. Trace:
I20260505 07:54:12.721002 1357 rpcz_store.cc:276] 0505 07:54:11.241297 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.689626 (+1448329us) service_pool.cc:224] Handling call
0505 07:54:12.690755 (+ 1129us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:12.691180 (+ 425us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:12.720505 (+ 29325us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:12.720639 (+ 134us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:12.720733 (+ 94us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.691625 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.691716 (+ 91us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.691720 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:12.691722 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:12.691808 (+ 86us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.691818 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.691824 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.691900 (+ 76us) tablet.cc:701] Row locks acquired
0505 07:54:12.691904 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:12.691933 (+ 29us) write_op.cc:270] Start()
0505 07:54:12.691957 (+ 24us) write_op.cc:276] Timestamp: P: 1777967652691823 usec, L: 0
0505 07:54:12.691962 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.692100 (+ 138us) log.cc:844] Serialized 433 byte log entry
0505 07:54:12.706682 (+ 14582us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.706798 (+ 116us) write_op.cc:301] APPLY: starting
0505 07:54:12.706816 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.706870 (+ 54us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.706930 (+ 60us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.707094 (+ 164us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.707135 (+ 41us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:12.707145 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:12.707223 (+ 78us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.707286 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.707304 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:12.707309 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.707378 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.707383 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":60,"apply.run_cpu_time_us":568,"apply.run_wall_time_us":568,"num_ops":2,"prepare.queue_time_us":40,"prepare.run_cpu_time_us":531,"prepare.run_wall_time_us":545,"replication_time_us":14688}]]}
I20260505 07:54:12.722059 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_1000a6ae9ea548fea1b484b91e275b82"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:12.722672 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1000a6ae9ea548fea1b484b91e275b82 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:12.723729 1181 tablet_bootstrap.cc:492] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.723925 1181 ts_tablet_manager.cc:1403] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.002s
I20260505 07:54:12.724645 1181 raft_consensus.cc:359] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.724892 1181 raft_consensus.cc:385] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.724996 1181 raft_consensus.cc:740] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.725229 1181 consensus_queue.cc:260] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.725708 1181 ts_tablet_manager.cc:1434] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:12.726006 1181 tablet_bootstrap.cc:492] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.727599 1181 tablet_bootstrap.cc:654] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.729027 1353 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 171) took 1441 ms. Trace:
I20260505 07:54:12.729128 1353 rpcz_store.cc:276] 0505 07:54:11.287682 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.693457 (+1405775us) service_pool.cc:224] Handling call
0505 07:54:12.711609 (+ 18152us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:12.712099 (+ 490us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:12.728765 (+ 16666us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:12.728876 (+ 111us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:12.728971 (+ 95us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.712617 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.712720 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.712724 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:12.712727 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:12.712821 (+ 94us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.712831 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.712867 (+ 36us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.712945 (+ 78us) tablet.cc:701] Row locks acquired
0505 07:54:12.712948 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:12.712977 (+ 29us) write_op.cc:270] Start()
0505 07:54:12.713006 (+ 29us) write_op.cc:276] Timestamp: P: 1777967652712863 usec, L: 0
0505 07:54:12.713012 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.713173 (+ 161us) log.cc:844] Serialized 433 byte log entry
0505 07:54:12.716850 (+ 3677us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.716936 (+ 86us) write_op.cc:301] APPLY: starting
0505 07:54:12.716947 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.716983 (+ 36us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.717003 (+ 20us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.717101 (+ 98us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.717134 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:12.717140 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:12.717194 (+ 54us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.717241 (+ 47us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.717255 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:12.717260 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.717321 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.717327 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":48,"apply.run_cpu_time_us":374,"apply.run_wall_time_us":373,"num_ops":2,"prepare.queue_time_us":55,"prepare.run_cpu_time_us":638,"prepare.run_wall_time_us":639,"raft.queue_time_us":57,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":252,"replication_time_us":3802}]]}
I20260505 07:54:12.729900 1353 master_service.cc:954] ReplaceTablet: received request to replace tablet b5d113a872f9400c8f28d0239ca42377 from {username='slave'} at 127.0.0.1:46452
W20260505 07:54:12.731318 991 consensus_peers.cc:597] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f121fdd001ec4ea38ccd24a6afd9e51c. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:12.743820 2650 raft_consensus.cc:493] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.743970 2650 raft_consensus.cc:515] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.744678 2650 leader_election.cc:290] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.744900 1181 tablet_bootstrap.cc:492] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.745033 1181 ts_tablet_manager.cc:1403] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20260505 07:54:12.745802 1181 raft_consensus.cc:359] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.746042 1181 raft_consensus.cc:385] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.746127 1181 raft_consensus.cc:740] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.746140 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba616702dc6040bb805bb3ffc8a8a871" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.746392 1181 consensus_queue.cc:260] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.746999 1181 ts_tablet_manager.cc:1434] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:12.747310 1181 tablet_bootstrap.cc:492] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.748939 1181 tablet_bootstrap.cc:654] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.760349 1353 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet b5d113a872f9400c8f28d0239ca42377
I20260505 07:54:12.761199 757 tablet_service.cc:1558] Processing DeleteTablet for tablet b5d113a872f9400c8f28d0239ca42377 with delete_type TABLET_DATA_DELETED (replaced by tablet 89d3e08134704859abc44257f484b8cd) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.761493 1181 tablet_bootstrap.cc:492] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.761693 1181 ts_tablet_manager.cc:1403] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260505 07:54:12.761723 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet b5d113a872f9400c8f28d0239ca42377 with delete_type TABLET_DATA_DELETED (replaced by tablet 89d3e08134704859abc44257f484b8cd) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:12.762393 1353 catalog_manager.cc:6741] ReplaceTablet: tablet b5d113a872f9400c8f28d0239ca42377 deleted and replaced by tablet 89d3e08134704859abc44257f484b8cd
I20260505 07:54:12.762565 1353 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 173) took 1380 ms. Trace:
I20260505 07:54:12.762724 887 tablet_service.cc:1558] Processing DeleteTablet for tablet b5d113a872f9400c8f28d0239ca42377 with delete_type TABLET_DATA_DELETED (replaced by tablet 89d3e08134704859abc44257f484b8cd) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:12.763089 1181 raft_consensus.cc:359] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.762727 1353 rpcz_store.cc:276] 0505 07:54:11.382264 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.729784 (+1347520us) service_pool.cc:224] Handling call
0505 07:54:12.762531 (+ 32747us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.740625 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.740739 (+ 114us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.740744 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:12.740746 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:12.740863 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.740879 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.740886 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.740988 (+ 102us) tablet.cc:701] Row locks acquired
0505 07:54:12.740991 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:12.741025 (+ 34us) write_op.cc:270] Start()
0505 07:54:12.741056 (+ 31us) write_op.cc:276] Timestamp: P: 1777967652741007 usec, L: 0
0505 07:54:12.741059 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.741262 (+ 203us) log.cc:844] Serialized 656 byte log entry
0505 07:54:12.748082 (+ 6820us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.748189 (+ 107us) write_op.cc:301] APPLY: starting
0505 07:54:12.748204 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.748252 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.748299 (+ 47us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.748397 (+ 98us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.748458 (+ 61us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:12.748468 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:12.748560 (+ 92us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.748621 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.748642 (+ 21us) write_op.cc:454] Released schema lock
0505 07:54:12.748650 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.748723 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.748728 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":60,"apply.run_cpu_time_us":522,"apply.run_wall_time_us":522,"num_ops":2,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":729,"prepare.run_wall_time_us":728,"raft.queue_time_us":66,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":210,"replication_time_us":6991}]]}
I20260505 07:54:12.763447 1353 master_service.cc:954] ReplaceTablet: received request to replace tablet c4dd3ece5cb34ceb82feeb67e75da0b5 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:12.763899 1181 raft_consensus.cc:385] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.764001 1181 raft_consensus.cc:740] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.764238 1181 consensus_queue.cc:260] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.764837 1181 ts_tablet_manager.cc:1434] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:12.765134 1181 tablet_bootstrap.cc:492] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.767155 1181 tablet_bootstrap.cc:654] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.767700 2268 ts_tablet_manager.cc:1929] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.767822 2268 log.cc:1199] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/36b33d265ba04e93993c7e0a4efea1a4
I20260505 07:54:12.768792 2268 ts_tablet_manager.cc:1950] T 36b33d265ba04e93993c7e0a4efea1a4 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.769181 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 137) took 3147 ms. Trace:
I20260505 07:54:12.769270 2268 rpcz_store.cc:276] 0505 07:54:09.622172 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.622222 (+ 50us) service_pool.cc:224] Handling call
0505 07:54:12.646434 (+3024212us) ts_tablet_manager.cc:1151] Deleting tablet 36b33d265ba04e93993c7e0a4efea1a4
0505 07:54:12.646465 (+ 31us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.705060 (+ 58595us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.767519 (+ 62459us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.769164 (+ 1645us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3024041}
I20260505 07:54:12.769582 2268 tablet_replica.cc:333] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:12.769833 2268 raft_consensus.cc:2243] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.770179 2268 raft_consensus.cc:2272] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.770776 2268 ts_tablet_manager.cc:1916] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.771858 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 36b33d265ba04e93993c7e0a4efea1a4 (table default.Table_0b3a73b640a644579fa864cce1a7ecbe [id=6c1e059a94014956ab0b1d6c750d8494]) successfully deleted
I20260505 07:54:12.778373 1357 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 172) took 1490 ms. Trace:
I20260505 07:54:12.778578 1357 rpcz_store.cc:276] 0505 07:54:11.287826 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.721892 (+1434066us) service_pool.cc:224] Handling call
0505 07:54:12.722817 (+ 925us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:12.723267 (+ 450us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:12.778129 (+ 54862us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:12.778237 (+ 108us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:12.778334 (+ 97us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.751552 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.751686 (+ 134us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.751690 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:12.751692 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:12.751772 (+ 80us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.751786 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.751793 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.751884 (+ 91us) tablet.cc:701] Row locks acquired
0505 07:54:12.751886 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:12.751913 (+ 27us) write_op.cc:270] Start()
0505 07:54:12.751937 (+ 24us) write_op.cc:276] Timestamp: P: 1777967652751896 usec, L: 0
0505 07:54:12.751940 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.752136 (+ 196us) log.cc:844] Serialized 433 byte log entry
0505 07:54:12.757318 (+ 5182us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.757420 (+ 102us) write_op.cc:301] APPLY: starting
0505 07:54:12.757432 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.757470 (+ 38us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.757492 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.757584 (+ 92us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.757621 (+ 37us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:12.757631 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:12.757702 (+ 71us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.757762 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.757782 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:12.757789 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.757863 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.757867 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":57,"apply.run_cpu_time_us":422,"apply.run_wall_time_us":422,"num_ops":2,"prepare.queue_time_us":36,"prepare.run_cpu_time_us":671,"prepare.run_wall_time_us":671,"raft.queue_time_us":54,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":203,"replication_time_us":5350}]]}
I20260505 07:54:12.779564 1357 master_service.cc:954] ReplaceTablet: received request to replace tablet 91d57f5bb6e847e6b099134d1382b730 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:12.780057 1357 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 175) took 1021 ms. Trace:
I20260505 07:54:12.780212 1357 rpcz_store.cc:276] 0505 07:54:11.758925 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.779496 (+1020571us) service_pool.cc:224] Handling call
0505 07:54:12.780006 (+ 510us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:12.780696 1357 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6d46a7a584f44bfbb6910845bb4b53a2" } modify_external_catalogs: true
I20260505 07:54:12.780835 1181 tablet_bootstrap.cc:492] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.780920 1357 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6d46a7a584f44bfbb6910845bb4b53a2" } modify_external_catalogs: true
I20260505 07:54:12.781016 1181 ts_tablet_manager.cc:1403] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.006s
I20260505 07:54:12.781716 1181 raft_consensus.cc:359] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.781926 1181 raft_consensus.cc:385] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.782020 1181 raft_consensus.cc:740] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.782413 1181 consensus_queue.cc:260] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.782932 1181 ts_tablet_manager.cc:1434] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:12.783210 1181 tablet_bootstrap.cc:492] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:12.784776 1181 tablet_bootstrap.cc:654] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.802857 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:12.803120 1353 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet c4dd3ece5cb34ceb82feeb67e75da0b5
I20260505 07:54:12.804774 757 tablet_service.cc:1558] Processing DeleteTablet for tablet c4dd3ece5cb34ceb82feeb67e75da0b5 with delete_type TABLET_DATA_DELETED (replaced by tablet d5b391d72d784c4db3457e464ad5ee93) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:12.807772 1172 tablet_bootstrap.cc:492] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:12.808048 1172 ts_tablet_manager.cc:1403] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.237s user 0.006s sys 0.019s
I20260505 07:54:12.809115 827 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:12.809506 1353 catalog_manager.cc:6741] ReplaceTablet: tablet c4dd3ece5cb34ceb82feeb67e75da0b5 deleted and replaced by tablet d5b391d72d784c4db3457e464ad5ee93
I20260505 07:54:12.809655 1353 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 174) took 1216 ms. Trace:
I20260505 07:54:12.809744 1353 rpcz_store.cc:276] 0505 07:54:11.592958 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.763347 (+1170389us) service_pool.cc:224] Handling call
0505 07:54:12.802826 (+ 39479us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.802833 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:12.809592 (+ 6759us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.764754 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.764882 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.764886 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:12.764888 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:12.764989 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.764999 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.765003 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.765095 (+ 92us) tablet.cc:701] Row locks acquired
0505 07:54:12.765097 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:12.765128 (+ 31us) write_op.cc:270] Start()
0505 07:54:12.765157 (+ 29us) write_op.cc:276] Timestamp: P: 1777967652765107 usec, L: 0
0505 07:54:12.765160 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.765340 (+ 180us) log.cc:844] Serialized 622 byte log entry
0505 07:54:12.775506 (+ 10166us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.775622 (+ 116us) write_op.cc:301] APPLY: starting
0505 07:54:12.775638 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.775678 (+ 40us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.775714 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.775811 (+ 97us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.775876 (+ 65us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:12.775884 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:12.775950 (+ 66us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.776003 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.776019 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:12.776026 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":71,"apply.run_cpu_time_us":460,"apply.run_wall_time_us":459,"num_ops":2,"prepare.queue_time_us":43,"prepare.run_cpu_time_us":665,"prepare.run_wall_time_us":666,"raft.queue_time_us":54,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":236,"replication_time_us":10314}]]}
I20260505 07:54:12.810746 1172 raft_consensus.cc:359] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.811205 1172 raft_consensus.cc:385] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.811348 1172 raft_consensus.cc:740] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.811791 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba616702dc6040bb805bb3ffc8a8a871" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.811982 767 raft_consensus.cc:2393] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:12.811933 1172 consensus_queue.cc:260] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.812323 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet c4dd3ece5cb34ceb82feeb67e75da0b5 with delete_type TABLET_DATA_DELETED (replaced by tablet d5b391d72d784c4db3457e464ad5ee93) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:12.812484 887 tablet_service.cc:1558] Processing DeleteTablet for tablet c4dd3ece5cb34ceb82feeb67e75da0b5 with delete_type TABLET_DATA_DELETED (replaced by tablet d5b391d72d784c4db3457e464ad5ee93) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:12.812827 2673 leader_election.cc:290] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.813299 1176 raft_consensus.cc:359] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.813457 1176 raft_consensus.cc:385] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.813508 1176 raft_consensus.cc:740] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.813506 1172 ts_tablet_manager.cc:1434] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20260505 07:54:12.813691 1176 consensus_queue.cc:260] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.814003 1172 tablet_bootstrap.cc:492] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:12.814181 1176 ts_tablet_manager.cc:1434] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.099s user 0.001s sys 0.000s
I20260505 07:54:12.814270 1047 raft_consensus.cc:3060] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.814409 1176 tablet_bootstrap.cc:492] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.816108 1176 tablet_bootstrap.cc:654] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.816756 2564 raft_consensus.cc:493] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.816861 2564 raft_consensus.cc:515] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.817350 2564 leader_election.cc:290] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.817716 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe19306f7df49fdb771d21f6560e697" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:12.817862 1051 raft_consensus.cc:3060] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.818460 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef22101fe014e21adea6e10f4b550e7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:12.819119 856 leader_election.cc:343] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.819686 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef22101fe014e21adea6e10f4b550e7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:12.820226 854 leader_election.cc:343] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.820318 854 leader_election.cc:304] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:12.822154 988 leader_election.cc:304] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:12.822503 988 consensus_peers.cc:597] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 17327cee69a24e0cb493ff55f948bf6c. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:12.825443 2564 raft_consensus.cc:2749] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.826433 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe19306f7df49fdb771d21f6560e697" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:12.826611 920 raft_consensus.cc:3060] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:12.827929 991 consensus_peers.cc:597] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1852f7f4ff594639908b03730b68fe86. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:12.828954 2672 raft_consensus.cc:493] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.829072 2672 raft_consensus.cc:515] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.829557 2672 leader_election.cc:290] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.830065 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df1b62a30b2842b897a04c793bb5e7ce" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:12.830235 1054 raft_consensus.cc:2393] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:12.832415 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df1b62a30b2842b897a04c793bb5e7ce" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.833019 718 leader_election.cc:304] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:12.833592 2673 raft_consensus.cc:3060] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.838650 1172 tablet_bootstrap.cc:654] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.844285 2672 consensus_queue.cc:1048] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:12.810544 1353 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_1a1a26d7818c4e038e590867f615d13f" } new_table_name: "default.Table_a2ac48b8b2a34b6580b27ea49735f549" modify_external_catalogs: true
I20260505 07:54:12.847211 2447 ts_tablet_manager.cc:1929] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.847306 2447 log.cc:1199] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/976359601ad74b149696dc7f555b550c
I20260505 07:54:12.847537 2268 ts_tablet_manager.cc:1929] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.847667 2268 log.cc:1199] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/d3d11bd19f074bbab101448b45426b7a
I20260505 07:54:12.848076 2268 ts_tablet_manager.cc:1950] T d3d11bd19f074bbab101448b45426b7a P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.849489 2447 ts_tablet_manager.cc:1950] T 976359601ad74b149696dc7f555b550c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:12.849828 1878 raft_consensus.cc:2749] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.850574 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 138) took 3228 ms. Trace:
I20260505 07:54:12.850651 2268 rpcz_store.cc:276] 0505 07:54:09.622546 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.622593 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:12.769527 (+3146934us) ts_tablet_manager.cc:1151] Deleting tablet d3d11bd19f074bbab101448b45426b7a
0505 07:54:12.769540 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.805829 (+ 36289us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.847484 (+ 41655us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.850559 (+ 3075us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3146813}
I20260505 07:54:12.850966 2268 tablet_replica.cc:333] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:12.851154 2268 raft_consensus.cc:2243] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:12.851287 2268 raft_consensus.cc:2272] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.851802 2268 ts_tablet_manager.cc:1916] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.852435 1051 raft_consensus.cc:2468] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:12.853011 717 leader_election.cc:304] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:12.853549 2564 consensus_queue.cc:1048] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:12.853636 920 raft_consensus.cc:2468] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:12.854310 2672 raft_consensus.cc:2804] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:12.854418 2672 raft_consensus.cc:697] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:12.854617 2672 consensus_queue.cc:237] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:12.855726 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet d3d11bd19f074bbab101448b45426b7a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]) successfully deleted
I20260505 07:54:12.858269 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 153) took 3367 ms. Trace:
I20260505 07:54:12.858371 2447 rpcz_store.cc:276] 0505 07:54:09.482543 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.482590 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:12.550779 (+3068189us) ts_tablet_manager.cc:1151] Deleting tablet 976359601ad74b149696dc7f555b550c
0505 07:54:12.550790 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.703963 (+153173us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.847150 (+143187us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.849914 (+ 2764us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3068034}
I20260505 07:54:12.858889 2447 tablet_replica.cc:333] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:12.858866 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 976359601ad74b149696dc7f555b550c (table default.Table_cb7435665bdd45ff8f517372a18d10b6 [id=0d5a336f16bb433fb2ee0f23189afc91]) successfully deleted
I20260505 07:54:12.861186 1176 tablet_bootstrap.cc:492] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.861299 1176 ts_tablet_manager.cc:1403] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.047s user 0.005s sys 0.000s
I20260505 07:54:12.862007 1176 raft_consensus.cc:359] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.862164 1176 raft_consensus.cc:385] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.862215 1176 raft_consensus.cc:740] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.862411 1176 consensus_queue.cc:260] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.862758 1176 ts_tablet_manager.cc:1434] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260505 07:54:12.864019 1176 tablet_bootstrap.cc:492] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.865732 2564 raft_consensus.cc:493] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.865837 2447 raft_consensus.cc:2243] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:12.865851 2564 raft_consensus.cc:515] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.866176 1176 tablet_bootstrap.cc:654] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.866394 2564 leader_election.cc:290] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.866783 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b256d072933b4566b5587224274deb66" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:12.866945 1047 raft_consensus.cc:2468] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:12.867399 856 leader_election.cc:304] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:12.867739 2564 raft_consensus.cc:2804] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.867806 2564 raft_consensus.cc:493] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.867861 2564 raft_consensus.cc:3060] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.875329 2505 raft_consensus.cc:493] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.875473 2505 raft_consensus.cc:515] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.876091 2505 leader_election.cc:290] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.876667 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e0c8ad716e443588d4ae3aa28ed4bf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:12.876842 1047 raft_consensus.cc:2468] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:12.877354 856 leader_election.cc:304] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:12.877627 2505 raft_consensus.cc:2804] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.877691 2505 raft_consensus.cc:493] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.877748 2505 raft_consensus.cc:3060] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.882552 2670 raft_consensus.cc:493] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.882571 2447 raft_consensus.cc:2272] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:12.882685 2670 raft_consensus.cc:515] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.883188 2447 ts_tablet_manager.cc:1916] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:12.883232 2670 leader_election.cc:290] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.884058 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b256d072933b4566b5587224274deb66" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.884758 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66082a7a75e345639e7a268727922fbb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:12.910702 2749 raft_consensus.cc:493] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.910950 2749 raft_consensus.cc:515] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.912309 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f106a6d4855148fe83b1e90ea9f27d39" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:12.912570 1047 raft_consensus.cc:2468] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:12.913076 856 leader_election.cc:304] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:12.917057 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66082a7a75e345639e7a268727922fbb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.917333 920 raft_consensus.cc:2393] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:12.917881 2749 leader_election.cc:290] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.918540 2749 raft_consensus.cc:2804] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.918704 2749 raft_consensus.cc:493] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.919314 2749 raft_consensus.cc:3060] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.921550 1181 tablet_bootstrap.cc:492] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:12.922309 718 leader_election.cc:304] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:12.923882 2670 raft_consensus.cc:3060] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:12.924422 718 consensus_peers.cc:597] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 54e4f7f895d943639653c8c32724baf2. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:12.924752 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f106a6d4855148fe83b1e90ea9f27d39" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:12.925122 718 consensus_peers.cc:597] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 003cd4472b424aacbba8c5f89900e228. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:12.925231 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e0c8ad716e443588d4ae3aa28ed4bf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:12.927408 854 leader_election.cc:343] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:12.932360 854 leader_election.cc:343] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:12.934979 854 leader_election.cc:343] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.937417 1172 tablet_bootstrap.cc:492] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:12.939523 1172 ts_tablet_manager.cc:1403] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.126s user 0.003s sys 0.002s
I20260505 07:54:12.941069 2674 raft_consensus.cc:493] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.941202 2674 raft_consensus.cc:515] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.941334 1176 tablet_bootstrap.cc:492] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.941453 1176 ts_tablet_manager.cc:1403] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.077s user 0.005s sys 0.005s
I20260505 07:54:12.941788 2674 leader_election.cc:290] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.941570 1172 raft_consensus.cc:359] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.941926 1172 raft_consensus.cc:385] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.942080 1172 raft_consensus.cc:740] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.942245 1176 raft_consensus.cc:359] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.942404 1176 raft_consensus.cc:385] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.942467 1176 raft_consensus.cc:740] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.942649 1176 consensus_queue.cc:260] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.942543 1172 consensus_queue.cc:260] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.943138 1176 ts_tablet_manager.cc:1434] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:12.943423 1176 tablet_bootstrap.cc:492] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:54:12.943562 717 leader_election.cc:343] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: e010f23c4df647ab863e03c3cfed7898
I20260505 07:54:12.943756 1172 ts_tablet_manager.cc:1434] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260505 07:54:12.946316 2505 raft_consensus.cc:515] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.946959 2505 leader_election.cc:290] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.947117 2673 raft_consensus.cc:2749] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:12.949683 2564 raft_consensus.cc:515] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.950368 2564 leader_election.cc:290] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.929853 1878 raft_consensus.cc:493] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.951751 1878 raft_consensus.cc:515] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.952440 1878 leader_election.cc:290] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:12.953080 1176 tablet_bootstrap.cc:654] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.953115 2268 ts_tablet_manager.cc:1929] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:12.953262 2268 log.cc:1199] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/4258df74ea0841449846cdf1f6e00b9b
I20260505 07:54:12.953680 2268 ts_tablet_manager.cc:1950] T 4258df74ea0841449846cdf1f6e00b9b P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:12.954275 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c837877946604ef29f95dac9dd449bf3" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.954450 920 raft_consensus.cc:2468] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:12.954790 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c837877946604ef29f95dac9dd449bf3" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:12.955853 991 leader_election.cc:304] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:12.956161 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:12.956224 2307 raft_consensus.cc:2804] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:12.956318 2307 raft_consensus.cc:493] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:12.956424 2307 raft_consensus.cc:3060] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:12.956553 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:12.956710 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:12.957029 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:12.957343 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:12.957511 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:12.957683 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:12.957877 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:12.958060 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:12.958364 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:12.958622 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:12.958865 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:12.959035 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:12.959193 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:12.959336 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:12.964686 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b256d072933b4566b5587224274deb66" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:12.965104 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e0c8ad716e443588d4ae3aa28ed4bf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:12.965241 781 raft_consensus.cc:3060] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:12.966585 854 leader_election.cc:343] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.966647 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e010f23c4df647ab863e03c3cfed7898" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:12.968012 920 raft_consensus.cc:2393] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:12.968115 1172 tablet_bootstrap.cc:492] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:12.969331 718 leader_election.cc:304] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:12.969647 2673 raft_consensus.cc:3060] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:12.970268 988 leader_election.cc:343] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:12.971122 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 58067.8us
I20260505 07:54:12.974210 1176 tablet_bootstrap.cc:492] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:12.974347 1176 ts_tablet_manager.cc:1403] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.000s
I20260505 07:54:12.976245 1181 ts_tablet_manager.cc:1403] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.193s user 0.008s sys 0.012s
I20260505 07:54:12.976306 1176 raft_consensus.cc:359] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.976480 1176 raft_consensus.cc:385] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:12.976543 1176 raft_consensus.cc:740] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:12.976732 1176 consensus_queue.cc:260] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.977542 1181 raft_consensus.cc:359] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.982157 1172 tablet_bootstrap.cc:654] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.978637 1176 ts_tablet_manager.cc:1434] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260505 07:54:12.983460 1176 tablet_bootstrap.cc:492] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:12.985714 2752 raft_consensus.cc:493] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:12.985857 2752 raft_consensus.cc:515] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:12.986364 2752 leader_election.cc:290] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:12.987530 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:12.987797 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:12.988955 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:12.990042 854 leader_election.cc:343] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:12.992154 1176 tablet_bootstrap.cc:654] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:12.989732 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:12.993160 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 90a13ca612f241fc97ea12124677a0b1 (table default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79]). Expected version 2 got 0
I20260505 07:54:12.993516 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:12.993799 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:12.994184 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 2 got 0
I20260505 07:54:12.995016 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:12.995255 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:12.995431 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:12.995932 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:12.996721 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:12.997207 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
W20260505 07:54:12.997283 717 consensus_peers.cc:597] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3d11bd19f074bbab101448b45426b7a. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:12.997648 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:12.998935 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:12.999228 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:12.999511 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:13.007341 2752 consensus_queue.cc:1048] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.008889 1176 tablet_bootstrap.cc:492] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.009030 1176 ts_tablet_manager.cc:1403] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.000s
W20260505 07:54:13.009616 717 consensus_peers.cc:597] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a79578757824bfc9df129e3475e7af3. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:13.009732 1172 tablet_bootstrap.cc:492] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:13.009856 1176 raft_consensus.cc:359] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.010035 1176 raft_consensus.cc:385] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.010016 1172 ts_tablet_manager.cc:1403] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.042s user 0.007s sys 0.000s
I20260505 07:54:13.010097 1176 raft_consensus.cc:740] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.010279 1176 consensus_queue.cc:260] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.010769 1176 ts_tablet_manager.cc:1434] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:12.999120 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7e0c8ad716e443588d4ae3aa28ed4bf" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:13.005213 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.012430 1176 tablet_bootstrap.cc:492] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.012492 1172 raft_consensus.cc:359] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.012830 1172 raft_consensus.cc:385] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.012938 1172 raft_consensus.cc:740] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.013447 1172 consensus_queue.cc:260] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.014194 1176 tablet_bootstrap.cc:654] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.015122 1047 raft_consensus.cc:3060] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.005470 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b256d072933b4566b5587224274deb66" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:13.016247 1054 raft_consensus.cc:3060] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.016664 1181 raft_consensus.cc:385] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.016808 1181 raft_consensus.cc:740] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.017133 1181 consensus_queue.cc:260] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.017987 1181 ts_tablet_manager.cc:1434] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.042s user 0.001s sys 0.001s
I20260505 07:54:13.018345 1181 tablet_bootstrap.cc:492] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:13.020824 1181 tablet_bootstrap.cc:654] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.021492 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:13.022275 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:13.022548 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:13.022784 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 21480c42b73e4bf0ab2453b085a867c0 (table default.Table_925028e721e24ba68f31fe1040d4c2fd [id=0faa6676bab04919984b580299143271]). Expected version 1 got 0
I20260505 07:54:13.023000 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:13.023208 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:13.023524 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:13.023836 1172 ts_tablet_manager.cc:1434] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.013s user 0.002s sys 0.000s
I20260505 07:54:13.024281 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:13.024560 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:13.025027 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:13.025192 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:13.025537 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
W20260505 07:54:13.026729 856 leader_election.cc:343] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.027475 856 leader_election.cc:304] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.027801 2753 raft_consensus.cc:2749] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:13.029536 1318 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 64ca343bd32a4eeb80901bc3157b3b6e consensus state has no leader
I20260505 07:54:13.033138 1176 tablet_bootstrap.cc:492] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.033305 1176 ts_tablet_manager.cc:1403] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20260505 07:54:13.033888 2564 raft_consensus.cc:493] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.034094 2564 raft_consensus.cc:515] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.034348 1176 raft_consensus.cc:359] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.034535 1176 raft_consensus.cc:385] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.034603 1176 raft_consensus.cc:740] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.034795 1176 consensus_queue.cc:260] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.035049 2564 leader_election.cc:290] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.035300 1176 ts_tablet_manager.cc:1434] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:13.035545 1176 tablet_bootstrap.cc:492] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.035799 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa049ad4edbd466a860428c822a52724" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:13.036463 854 leader_election.cc:343] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.037132 1176 tablet_bootstrap.cc:654] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.030265 2650 raft_consensus.cc:493] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.031850 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 139) took 3409 ms. Trace:
I20260505 07:54:13.037261 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa049ad4edbd466a860428c822a52724" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.041298 2268 rpcz_store.cc:276] 0505 07:54:09.622868 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.622922 (+ 54us) service_pool.cc:224] Handling call
0505 07:54:12.850901 (+3227979us) ts_tablet_manager.cc:1151] Deleting tablet 4258df74ea0841449846cdf1f6e00b9b
0505 07:54:12.850908 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.943985 (+ 93077us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:12.953061 (+ 9076us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.031833 (+ 78772us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3227850}
I20260505 07:54:13.041478 1050 raft_consensus.cc:2468] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:13.041736 2268 tablet_replica.cc:333] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:13.041975 2268 raft_consensus.cc:2243] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:13.042029 856 leader_election.cc:304] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.042246 2564 raft_consensus.cc:2804] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.042320 2564 raft_consensus.cc:493] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.042309 2268 raft_consensus.cc:2272] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.042409 2564 raft_consensus.cc:3060] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.042757 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 4258df74ea0841449846cdf1f6e00b9b (table default.Table_b3b9364eea524724b176de87f62eb49e [id=1fd77c8cfe9d40dab5da490774294796]) successfully deleted
I20260505 07:54:13.041299 2650 raft_consensus.cc:515] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.043223 2268 ts_tablet_manager.cc:1916] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.043462 2650 leader_election.cc:290] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:13.044770 991 leader_election.cc:343] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: a7fe9c7199cc4039ae877dc3d4aab17e
I20260505 07:54:13.044962 2276 ts_tablet_manager.cc:1929] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.044991 2447 ts_tablet_manager.cc:1929] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.045065 2276 log.cc:1199] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/12501fdf2cc5491cb1a06db490e70810
I20260505 07:54:13.045094 2447 log.cc:1199] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/3d6494590052459485059f5dce46f34f
I20260505 07:54:13.045130 2670 raft_consensus.cc:2749] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:13.045395 2276 ts_tablet_manager.cc:1950] T 12501fdf2cc5491cb1a06db490e70810 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:13.045457 2447 ts_tablet_manager.cc:1950] T 3d6494590052459485059f5dce46f34f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:13.045819 2749 raft_consensus.cc:515] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.046365 2749 leader_election.cc:290] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.048740 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 154) took 3565 ms. Trace:
I20260505 07:54:13.048820 2447 rpcz_store.cc:276] 0505 07:54:09.482882 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.482940 (+ 58us) service_pool.cc:224] Handling call
0505 07:54:12.858844 (+3375904us) ts_tablet_manager.cc:1151] Deleting tablet 3d6494590052459485059f5dce46f34f
0505 07:54:12.858854 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.947390 (+ 88536us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.044924 (+ 97534us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.048722 (+ 3798us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23055,"tablet-delete.queue_time_us":3375762}
I20260505 07:54:13.049147 2447 tablet_replica.cc:333] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:13.049347 2447 raft_consensus.cc:2243] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:13.049487 2447 raft_consensus.cc:2272] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.050072 2447 ts_tablet_manager.cc:1916] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.050882 781 raft_consensus.cc:2468] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:13.053007 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 3d6494590052459485059f5dce46f34f (table default.Table_77b80258ea294c47bf9a6f1adb45c178 [id=85e91b69517e415194bc6c0dd62ccf9f]) successfully deleted
I20260505 07:54:13.055649 2307 raft_consensus.cc:515] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.056344 2307 leader_election.cc:290] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.060039 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c837877946604ef29f95dac9dd449bf3" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:13.064054 988 leader_election.cc:343] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.064741 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 83) took 5352 ms. Trace:
I20260505 07:54:13.064831 2276 rpcz_store.cc:276] 0505 07:54:07.712417 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.713244 (+ 827us) service_pool.cc:224] Handling call
0505 07:54:12.571127 (+4857883us) ts_tablet_manager.cc:1151] Deleting tablet 12501fdf2cc5491cb1a06db490e70810
0505 07:54:12.571135 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:12.576807 (+ 5672us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.044905 (+468098us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.064698 (+ 19793us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4857767}
I20260505 07:54:13.065140 2276 tablet_replica.cc:333] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:13.065317 2276 raft_consensus.cc:2243] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:13.065461 2276 raft_consensus.cc:2272] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.065968 2276 ts_tablet_manager.cc:1916] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.067528 854 leader_election.cc:304] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.067898 2749 raft_consensus.cc:2804] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:13.068017 2749 raft_consensus.cc:697] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:13.068392 1176 tablet_bootstrap.cc:492] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.068284 2749 consensus_queue.cc:237] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.068496 1176 ts_tablet_manager.cc:1403] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.033s user 0.000s sys 0.003s
I20260505 07:54:13.069006 1172 tablet_bootstrap.cc:492] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:13.069307 1176 raft_consensus.cc:359] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.069461 1176 raft_consensus.cc:385] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.069519 1176 raft_consensus.cc:740] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.069705 1176 consensus_queue.cc:260] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.069985 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f106a6d4855148fe83b1e90ea9f27d39" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:13.070223 1176 ts_tablet_manager.cc:1434] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:13.070482 1176 tablet_bootstrap.cc:492] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.072002 2307 raft_consensus.cc:493] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.072312 2307 raft_consensus.cc:515] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.072671 1176 tablet_bootstrap.cc:654] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.073256 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d86b8b4dadf4fd2b6409a8cb70e313e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.073678 2307 leader_election.cc:290] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:13.073843 988 leader_election.cc:343] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:13.074620 854 leader_election.cc:343] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.076582 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c837877946604ef29f95dac9dd449bf3" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:13.076774 920 raft_consensus.cc:3060] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.077512 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d86b8b4dadf4fd2b6409a8cb70e313e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.077699 918 raft_consensus.cc:2468] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:13.078614 991 leader_election.cc:304] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.079090 2307 raft_consensus.cc:2804] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.079241 2307 raft_consensus.cc:493] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.079403 2307 raft_consensus.cc:3060] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.084921 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7fe9c7199cc4039ae877dc3d4aab17e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:13.085664 988 leader_election.cc:343] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.085839 988 leader_election.cc:304] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.086059 1878 raft_consensus.cc:2749] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:13.092382 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f106a6d4855148fe83b1e90ea9f27d39" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:13.092658 1051 raft_consensus.cc:3060] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.093315 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 12501fdf2cc5491cb1a06db490e70810 (table default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]) successfully deleted
I20260505 07:54:13.094049 2505 consensus_queue.cc:1048] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.096278 2752 consensus_queue.cc:1048] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.102540 2672 raft_consensus.cc:493] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.102682 2672 raft_consensus.cc:515] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.104008 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84f464e83cb412eade9978ea99f6685" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.104214 918 raft_consensus.cc:2393] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:13.104563 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f84f464e83cb412eade9978ea99f6685" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.105283 717 leader_election.cc:304] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:13.105576 2674 raft_consensus.cc:3060] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.105854 2672 leader_election.cc:290] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.108700 1172 tablet_bootstrap.cc:654] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.146107 2673 raft_consensus.cc:2749] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:13.146507 1047 raft_consensus.cc:2468] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:13.146970 2564 raft_consensus.cc:515] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.147650 2564 leader_election.cc:290] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.150307 2674 raft_consensus.cc:2749] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:13.153699 2447 ts_tablet_manager.cc:1929] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.153820 2447 log.cc:1199] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/4783dd0b2fbe40219852d850fc0d9a82
I20260505 07:54:13.154265 2447 ts_tablet_manager.cc:1950] T 4783dd0b2fbe40219852d850fc0d9a82 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:13.154798 1176 tablet_bootstrap.cc:492] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.154923 1176 ts_tablet_manager.cc:1403] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.085s user 0.000s sys 0.003s
I20260505 07:54:13.155651 1176 raft_consensus.cc:359] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.155793 1176 raft_consensus.cc:385] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.155841 1176 raft_consensus.cc:740] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.156032 1176 consensus_queue.cc:260] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.156471 1176 ts_tablet_manager.cc:1434] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260505 07:54:13.156742 1176 tablet_bootstrap.cc:492] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.158852 1176 tablet_bootstrap.cc:654] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.163702 2276 ts_tablet_manager.cc:1929] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.163821 2276 log.cc:1199] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/f1058fcb9fa6457a85489049ca1d9ada
I20260505 07:54:13.164167 2276 ts_tablet_manager.cc:1950] T f1058fcb9fa6457a85489049ca1d9ada P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:13.168744 771 raft_consensus.cc:3060] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.171684 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa049ad4edbd466a860428c822a52724" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:13.171948 1047 raft_consensus.cc:3060] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.175944 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa049ad4edbd466a860428c822a52724" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:13.176117 772 raft_consensus.cc:3060] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.178757 1172 tablet_bootstrap.cc:492] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:13.179032 1172 ts_tablet_manager.cc:1403] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.116s user 0.003s sys 0.004s
I20260505 07:54:13.180620 1172 raft_consensus.cc:359] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.180954 1172 raft_consensus.cc:385] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.181048 1172 raft_consensus.cc:740] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.181438 1172 consensus_queue.cc:260] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.182603 1172 ts_tablet_manager.cc:1434] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260505 07:54:13.183686 2405 raft_consensus.cc:493] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.185073 1172 tablet_bootstrap.cc:492] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:13.186882 2405 raft_consensus.cc:515] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.187505 2405 leader_election.cc:290] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.188091 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b406c4c6ba741bc85eef54c6f576459" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.188280 781 raft_consensus.cc:2468] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:13.190057 988 leader_election.cc:304] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
W20260505 07:54:13.190666 991 leader_election.cc:343] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8b406c4c6ba741bc85eef54c6f576459
I20260505 07:54:13.191128 1878 raft_consensus.cc:2804] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.191241 1878 raft_consensus.cc:493] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.191205 2650 consensus_queue.cc:1048] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.191349 1878 raft_consensus.cc:3060] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:13.201213 991 consensus_peers.cc:597] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 909306e1e5f34b7c8fbc0d86cafce531. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:13.207005 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:13.210163 1181 tablet_bootstrap.cc:492] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:13.211642 1176 tablet_bootstrap.cc:492] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.211818 1176 ts_tablet_manager.cc:1403] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.001s
I20260505 07:54:13.212678 1176 raft_consensus.cc:359] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.212857 1176 raft_consensus.cc:385] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.212920 1176 raft_consensus.cc:740] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.213109 1176 consensus_queue.cc:260] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.213882 1176 ts_tablet_manager.cc:1434] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:13.214262 1181 ts_tablet_manager.cc:1403] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.196s user 0.009s sys 0.007s
I20260505 07:54:13.216423 1176 tablet_bootstrap.cc:492] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.217512 1181 raft_consensus.cc:359] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.218847 2670 raft_consensus.cc:493] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.218968 2670 raft_consensus.cc:515] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.219513 2670 leader_election.cc:290] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.223250 1172 tablet_bootstrap.cc:654] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.224004 1176 tablet_bootstrap.cc:654] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.224283 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f869abaa31e414ea0625e475db08e64" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.224490 918 raft_consensus.cc:2468] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:13.225436 718 leader_election.cc:304] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.226023 2414 raft_consensus.cc:2804] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.226101 2414 raft_consensus.cc:493] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.226161 2414 raft_consensus.cc:3060] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.228791 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 155) took 3745 ms. Trace:
I20260505 07:54:13.228885 2447 rpcz_store.cc:276] 0505 07:54:09.483210 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.483255 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:13.049101 (+3565846us) ts_tablet_manager.cc:1151] Deleting tablet 4783dd0b2fbe40219852d850fc0d9a82
0505 07:54:13.049111 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.146064 (+ 96953us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.153646 (+ 7582us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.228773 (+ 75127us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3565661}
I20260505 07:54:13.229461 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 4783dd0b2fbe40219852d850fc0d9a82 (table default.Table_0555c54d8e394cb1a8509950422c9ac8 [id=e59beab126684f548325a29eb663177b]) successfully deleted
I20260505 07:54:13.229233 2447 tablet_replica.cc:333] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:13.230216 2447 raft_consensus.cc:2243] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:13.230422 2447 raft_consensus.cc:2272] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.230912 2447 ts_tablet_manager.cc:1916] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.221486 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f869abaa31e414ea0625e475db08e64" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.232635 1050 raft_consensus.cc:2468] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:13.238891 2752 raft_consensus.cc:493] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.239078 2268 ts_tablet_manager.cc:1929] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:13.239028 2752 raft_consensus.cc:515] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.239166 2268 log.cc:1199] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/43e4c92b7c3140fdb2d6a290aaf0ad55
I20260505 07:54:13.239480 2268 ts_tablet_manager.cc:1950] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:13.239655 2752 leader_election.cc:290] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.243269 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5083f98e8e14ccf99a19cdf2bc59018" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.244042 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5083f98e8e14ccf99a19cdf2bc59018" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:13.244596 856 leader_election.cc:343] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:13.245034 854 leader_election.cc:343] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.245136 854 leader_election.cc:304] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.247042 2447 ts_tablet_manager.cc:1929] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:13.247089 2749 raft_consensus.cc:2749] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:13.247155 2447 log.cc:1199] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/8b406c4c6ba741bc85eef54c6f576459
I20260505 07:54:13.247496 2447 ts_tablet_manager.cc:1950] T 8b406c4c6ba741bc85eef54c6f576459 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:13.247714 1054 raft_consensus.cc:2468] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:13.250581 920 raft_consensus.cc:2468] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:13.250756 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 84) took 5538 ms. Trace:
I20260505 07:54:13.250876 2276 rpcz_store.cc:276] 0505 07:54:07.712591 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.713390 (+ 799us) service_pool.cc:224] Handling call
0505 07:54:13.065101 (+5351711us) ts_tablet_manager.cc:1151] Deleting tablet f1058fcb9fa6457a85489049ca1d9ada
0505 07:54:13.065109 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.146112 (+ 81003us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.163640 (+ 17528us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.250743 (+ 87103us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5351606}
I20260505 07:54:13.251219 2276 tablet_replica.cc:333] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:13.251416 2276 raft_consensus.cc:2243] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:13.251623 2276 raft_consensus.cc:2272] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.251767 2650 consensus_queue.cc:1048] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.252228 2276 ts_tablet_manager.cc:1916] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.252632 856 leader_election.cc:304] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.253114 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet f1058fcb9fa6457a85489049ca1d9ada (table default.Table_2872bda04bf547648372a8f4bf5dc3ce [id=42ae40c0861e4489b724c9b8e3b590d7]) successfully deleted
I20260505 07:54:13.253383 2749 raft_consensus.cc:2804] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:13.253504 2749 raft_consensus.cc:697] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:13.253733 2749 consensus_queue.cc:237] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.257845 2673 raft_consensus.cc:493] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.257997 2673 raft_consensus.cc:515] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.258172 991 leader_election.cc:304] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.258663 2302 raft_consensus.cc:2804] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:13.258786 2302 raft_consensus.cc:697] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:13.258826 2673 leader_election.cc:290] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.258965 2673 raft_consensus.cc:493] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.259053 2673 raft_consensus.cc:515] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.259023 2302 consensus_queue.cc:237] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.259534 2673 leader_election.cc:290] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.263229 2650 consensus_queue.cc:1048] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:13.265033 718 leader_election.cc:343] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: a7d8265c05b74b28959d0de41fad1fde
I20260505 07:54:13.273907 1176 tablet_bootstrap.cc:492] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.274057 1176 ts_tablet_manager.cc:1403] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.001s
I20260505 07:54:13.275300 1176 raft_consensus.cc:359] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.275473 1176 raft_consensus.cc:385] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.275527 1176 raft_consensus.cc:740] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.275763 1176 consensus_queue.cc:260] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.276278 1176 ts_tablet_manager.cc:1434] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:13.276552 1176 tablet_bootstrap.cc:492] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.276532 2742 raft_consensus.cc:493] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.276677 2742 raft_consensus.cc:515] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.277793 2742 leader_election.cc:290] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.279729 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe5f6f0774545b296bf1b03a08e145d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.279950 920 raft_consensus.cc:2468] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:13.280289 1176 tablet_bootstrap.cc:654] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.280541 991 leader_election.cc:304] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:13.281199 2650 raft_consensus.cc:2804] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.281304 2650 raft_consensus.cc:493] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.281440 2650 raft_consensus.cc:3060] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.281813 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=595 overwritten=0 applied=589 ignored=0} inserts{seen=554 ignored=0} mutations{seen=676 ignored=0} orphaned_commits=0. Pending: 6 replicates
I20260505 07:54:13.282058 2752 raft_consensus.cc:493] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:13.282212 2752 raft_consensus.cc:515] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:13.282781 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Bootstrap complete.
I20260505 07:54:13.282845 2752 leader_election.cc:290] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:13.283631 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
W20260505 07:54:13.284699 854 leader_election.cc:343] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: f88a5c0d0e2c436c8d8b6afc1f3c2edf
I20260505 07:54:13.286036 2749 raft_consensus.cc:493] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:13.286160 2749 raft_consensus.cc:515] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:13.286724 2749 leader_election.cc:290] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:13.288164 854 leader_election.cc:343] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 6ca2852cee84476dbe5e5072a43087a1
I20260505 07:54:13.289963 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe5f6f0774545b296bf1b03a08e145d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.290131 781 raft_consensus.cc:2468] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:13.290429 2752 raft_consensus.cc:493] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.290542 2752 raft_consensus.cc:515] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.291098 2752 leader_election.cc:290] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.291842 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9b7c6e9463342a5a73af12da4447f89" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:13.292309 854 leader_election.cc:343] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.293202 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9b7c6e9463342a5a73af12da4447f89" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:13.294158 856 leader_election.cc:343] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: f88a5c0d0e2c436c8d8b6afc1f3c2edf
I20260505 07:54:13.294288 856 leader_election.cc:304] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.294775 2564 raft_consensus.cc:2749] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:13.297115 856 leader_election.cc:343] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.297214 856 leader_election.cc:304] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:13.297541 856 leader_election.cc:343] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 6ca2852cee84476dbe5e5072a43087a1
I20260505 07:54:13.297629 856 leader_election.cc:304] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.298018 2564 raft_consensus.cc:2749] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:13.298182 2564 raft_consensus.cc:2749] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:13.303097 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:13.303380 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:13.303689 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:13.304003 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:13.304318 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:13.304661 2257 raft_consensus.cc:359] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 1 FOLLOWER]: Replica starting. Triggering 6 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:54:13.306043 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:13.308339 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:13.309110 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:13.311327 1176 tablet_bootstrap.cc:492] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.311700 1176 ts_tablet_manager.cc:1403] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.000s
I20260505 07:54:13.311975 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:13.313585 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:13.313369 1176 raft_consensus.cc:359] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.313809 1176 raft_consensus.cc:385] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.313958 1176 raft_consensus.cc:740] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.314422 1176 consensus_queue.cc:260] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.314965 1351 catalog_manager.cc:5671] T e1a403e4708e4725899c1cd5b9fb0e8b P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:13.315557 1176 ts_tablet_manager.cc:1434] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260505 07:54:13.316119 1176 tablet_bootstrap.cc:492] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.319412 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:13.319880 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e99f96a753047348371bee5c022684e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.320070 907 raft_consensus.cc:2468] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:13.320468 718 leader_election.cc:304] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.320789 2670 raft_consensus.cc:2804] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:13.320907 2670 raft_consensus.cc:493] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:13.321033 2670 raft_consensus.cc:3060] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:13.326979 2257 raft_consensus.cc:740] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d3eda04f9c47ae976d19c218b6dcc5, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.327025 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:13.327811 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:13.327937 2257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 589, Last appended: 1.595, Last appended by leader: 595, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:54:13.328998 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_00e64df3348c430399b8bb1afc7f44ee [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 1 got 0
I20260505 07:54:13.333025 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:13.333966 1176 tablet_bootstrap.cc:654] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.338585 2257 sys_catalog.cc:565] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260505 07:54:13.339066 2168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 3 candidate_status { last_received { term: 2 index: 686 } } ignore_live_leader: false dest_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" is_pre_election: true
I20260505 07:54:13.339208 2307 raft_consensus.cc:515] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:13.339736 2168 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37712 (request call id 1) took 3435 ms (client timeout 608 ms). Trace:
I20260505 07:54:13.339865 2307 leader_election.cc:290] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:13.339865 2168 rpcz_store.cc:269] 0505 07:54:09.904455 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.904518 (+ 63us) service_pool.cc:224] Handling call
0505 07:54:13.339685 (+3435167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:13.340821 2172 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 3 candidate_status { last_received { term: 2 index: 686 } } ignore_live_leader: false dest_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" is_pre_election: true
W20260505 07:54:13.341123 2172 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37712 (request call id 0) took 3923 ms (client timeout 608 ms). Trace:
W20260505 07:54:13.341241 2172 rpcz_store.cc:269] 0505 07:54:09.417863 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.417919 (+ 56us) service_pool.cc:224] Handling call
0505 07:54:13.341096 (+3923177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:13.342459 1051 raft_consensus.cc:2468] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:13.343284 2173 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:13.344702 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:13.345094 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]). Expected version 1 got 0
I20260505 07:54:13.345278 2276 ts_tablet_manager.cc:1929] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.345441 2276 log.cc:1199] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/141665fcd59048bf926acd0287f1853e
I20260505 07:54:13.345880 2276 ts_tablet_manager.cc:1950] T 141665fcd59048bf926acd0287f1853e P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:13.346875 772 raft_consensus.cc:2468] T aa049ad4edbd466a860428c822a52724 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:13.347504 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 156) took 3856 ms. Trace:
I20260505 07:54:13.347611 2447 rpcz_store.cc:276] 0505 07:54:09.490808 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.490868 (+ 60us) service_pool.cc:224] Handling call
0505 07:54:13.229186 (+3738318us) ts_tablet_manager.cc:1151] Deleting tablet 8b406c4c6ba741bc85eef54c6f576459
0505 07:54:13.229196 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.239002 (+ 9806us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.246984 (+ 7982us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.347488 (+100504us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3737954}
I20260505 07:54:13.347973 2447 tablet_replica.cc:333] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:13.348068 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:13.348158 2447 raft_consensus.cc:2243] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:13.348477 2447 raft_consensus.cc:2272] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.349090 2447 ts_tablet_manager.cc:1916] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.350975 2742 consensus_queue.cc:1048] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.351837 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 8b406c4c6ba741bc85eef54c6f576459 (table default.Table_d4d6e315fb8340e9b629d876d050ef05 [id=5fdbc0ad9c3f4e0a9b2fc0997a87870b]) successfully deleted
I20260505 07:54:13.352025 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:13.354390 854 leader_election.cc:304] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.355062 2749 raft_consensus.cc:2804] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:13.355077 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:13.355235 2749 raft_consensus.cc:697] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:13.355499 2749 consensus_queue.cc:237] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.355645 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:13.356021 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:13.358991 2779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:54:13.359282 2779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:54:13.363878 2674 raft_consensus.cc:493] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.364051 2674 raft_consensus.cc:515] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.364584 1176 tablet_bootstrap.cc:492] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.364751 1176 ts_tablet_manager.cc:1403] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.049s user 0.002s sys 0.001s
I20260505 07:54:13.365272 2674 leader_election.cc:290] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.365785 1176 raft_consensus.cc:359] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.365993 1176 raft_consensus.cc:385] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.366055 1176 raft_consensus.cc:740] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.366269 1176 consensus_queue.cc:260] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.366909 1176 ts_tablet_manager.cc:1434] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:13.367213 1176 tablet_bootstrap.cc:492] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.367224 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f106a6d4855148fe83b1e90ea9f27d39" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.367463 918 raft_consensus.cc:2393] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:13.367646 967 heartbeater.cc:344] Connected to a master server at 127.0.105.60:35401
I20260505 07:54:13.368691 2752 raft_consensus.cc:493] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.368911 2752 raft_consensus.cc:515] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.369145 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:13.369544 2752 leader_election.cc:290] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.369879 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:13.370999 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:13.371170 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66be7b6c33a342d6890e459d90d40583" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:13.372124 854 leader_election.cc:343] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.372529 1176 tablet_bootstrap.cc:654] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.374292 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:13.377730 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:13.378782 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66be7b6c33a342d6890e459d90d40583" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.379325 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:13.379880 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:13.381357 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
W20260505 07:54:13.381953 988 consensus_peers.cc:597] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4783dd0b2fbe40219852d850fc0d9a82. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:13.382225 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d86b8b4dadf4fd2b6409a8cb70e313e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:13.382493 772 raft_consensus.cc:3060] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.384699 856 leader_election.cc:304] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:13.385479 856 leader_election.cc:343] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.385671 856 leader_election.cc:304] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.385980 2564 raft_consensus.cc:2804] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:13.386099 2564 raft_consensus.cc:697] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:13.386416 2564 consensus_queue.cc:237] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.387953 2505 raft_consensus.cc:2749] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:13.389024 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:13.389317 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:13.389619 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:13.390398 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:13.392081 1351 catalog_manager.cc:5671] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:13.393350 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d86b8b4dadf4fd2b6409a8cb70e313e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:13.393581 907 raft_consensus.cc:3060] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.394032 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:13.394608 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:13.395296 1176 tablet_bootstrap.cc:492] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.395483 1176 ts_tablet_manager.cc:1403] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.002s
I20260505 07:54:13.396548 1176 raft_consensus.cc:359] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.396824 1176 raft_consensus.cc:385] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.396986 1176 raft_consensus.cc:740] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.397295 1176 consensus_queue.cc:260] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.398084 1176 ts_tablet_manager.cc:1434] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:13.398243 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 85) took 5685 ms. Trace:
I20260505 07:54:13.398453 1176 tablet_bootstrap.cc:492] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.398368 2276 rpcz_store.cc:276] 0505 07:54:07.712686 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.713518 (+ 832us) service_pool.cc:224] Handling call
0505 07:54:13.251170 (+5537652us) ts_tablet_manager.cc:1151] Deleting tablet 141665fcd59048bf926acd0287f1853e
0505 07:54:13.251181 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.339153 (+ 87972us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.345217 (+ 6064us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.398223 (+ 53006us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5537521}
I20260505 07:54:13.398862 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 86) took 5686 ms. Trace:
I20260505 07:54:13.398988 2276 rpcz_store.cc:276] 0505 07:54:07.712790 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.713674 (+ 884us) service_pool.cc:224] Handling call
0505 07:54:13.398739 (+5685065us) ts_tablet_manager.cc:1151] Deleting tablet d896e290167b4d0081e8e8b831c8ae2a
0505 07:54:13.398751 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.398846 (+ 95us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5684970}
I20260505 07:54:13.399361 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 87) took 5631 ms. Trace:
I20260505 07:54:13.399463 2276 rpcz_store.cc:276] 0505 07:54:07.768052 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.768106 (+ 54us) service_pool.cc:224] Handling call
0505 07:54:13.399249 (+5631143us) ts_tablet_manager.cc:1151] Deleting tablet abaa596871bb4a82aefc317932e91d35
0505 07:54:13.399259 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.399344 (+ 85us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5600537}
I20260505 07:54:13.398906 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 141665fcd59048bf926acd0287f1853e (table default.Table_0ec687e90aea4825b28b36eda22678fe [id=c551b1aa1f2e4eaab4ce7c6727c05465]) successfully deleted
I20260505 07:54:13.399881 2276 tablet_replica.cc:333] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:13.400063 2276 raft_consensus.cc:2243] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:13.400105 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet d896e290167b4d0081e8e8b831c8ae2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d896e290167b4d0081e8e8b831c8ae2a already in progress: creating tablet
I20260505 07:54:13.400219 2276 raft_consensus.cc:2272] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:13.400352 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet abaa596871bb4a82aefc317932e91d35 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abaa596871bb4a82aefc317932e91d35 already in progress: creating tablet
I20260505 07:54:13.400890 2276 ts_tablet_manager.cc:1916] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.404256 1176 tablet_bootstrap.cc:654] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.405205 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:13.405534 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:13.405794 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:13.406164 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:13.406430 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
W20260505 07:54:13.409293 1353 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 090fbe6c480249338e7210050770adeb consensus state has no leader
I20260505 07:54:13.409515 1353 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 177) took 1153 ms. Trace:
I20260505 07:54:13.409613 1353 rpcz_store.cc:276] 0505 07:54:12.255987 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.810363 (+554376us) service_pool.cc:224] Handling call
0505 07:54:12.846193 (+ 35830us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:12.846430 (+ 237us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:12.846631 (+ 201us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:12.846708 (+ 77us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:12.884402 (+ 37694us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.884407 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:12.884501 (+ 94us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:13.409453 (+524952us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.874623 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.874761 (+ 138us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.874769 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:12.874772 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:12.874903 (+ 131us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.874917 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.874922 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:12.874988 (+ 66us) tablet.cc:701] Row locks acquired
0505 07:54:12.874990 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:12.875044 (+ 54us) write_op.cc:270] Start()
0505 07:54:12.875078 (+ 34us) write_op.cc:276] Timestamp: P: 1777967652875011 usec, L: 0
0505 07:54:12.875083 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.875262 (+ 179us) log.cc:844] Serialized 405 byte log entry
0505 07:54:12.879798 (+ 4536us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.879945 (+ 147us) write_op.cc:301] APPLY: starting
0505 07:54:12.879961 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.880016 (+ 55us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.880047 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.880096 (+ 49us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.880131 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:12.880140 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:12.880230 (+ 90us) log.cc:844] Serialized 23 byte log entry
0505 07:54:12.880290 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.880310 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:12.880318 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":83,"apply.run_cpu_time_us":432,"apply.run_wall_time_us":432,"num_ops":1,"prepare.queue_time_us":50,"prepare.run_cpu_time_us":737,"prepare.run_wall_time_us":737,"raft.queue_time_us":767,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":229,"replication_time_us":4689}]]}
I20260505 07:54:13.412392 2674 raft_consensus.cc:493] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.412608 2674 raft_consensus.cc:515] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.413750 2674 leader_election.cc:290] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.414779 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a0cbf9c3d74cea92920189da332c1c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.415048 918 raft_consensus.cc:2468] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:13.415550 718 leader_election.cc:304] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.415978 2674 raft_consensus.cc:2804] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.416106 2674 raft_consensus.cc:493] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.416244 2674 raft_consensus.cc:3060] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.416704 1181 raft_consensus.cc:385] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.416846 1181 raft_consensus.cc:740] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.417119 1181 consensus_queue.cc:260] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.417884 1181 ts_tablet_manager.cc:1434] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.203s user 0.001s sys 0.001s
I20260505 07:54:13.418303 1181 tablet_bootstrap.cc:492] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:54:13.423893 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 68e8f9f7b30f4912af916c15f9ae25d8 consensus state has no leader
I20260505 07:54:13.424676 1352 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 571257bbb7e94315b60d84d6aa4917b1
I20260505 07:54:13.425310 1352 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 158) took 2470 ms. Trace:
I20260505 07:54:13.425683 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 571257bbb7e94315b60d84d6aa4917b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:12 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:13.425442 1352 rpcz_store.cc:276] 0505 07:54:10.955018 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.058832 (+1103814us) service_pool.cc:224] Handling call
0505 07:54:12.245529 (+186697us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:12.245683 (+ 154us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:12.245755 (+ 72us) catalog_manager.cc:2824] Locking tablets
0505 07:54:12.335579 (+ 89824us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:12.389376 (+ 53797us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:12.389450 (+ 74us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:13.424614 (+1035164us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:13.425258 (+ 644us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.336384 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.336546 (+ 162us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.336555 (+ 9us) write_op.cc:435] Acquired schema lock
0505 07:54:12.336560 (+ 5us) tablet.cc:662] Decoding operations
0505 07:54:12.336726 (+ 166us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.336742 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.336749 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.336882 (+ 133us) tablet.cc:701] Row locks acquired
0505 07:54:12.336885 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:12.336923 (+ 38us) write_op.cc:270] Start()
0505 07:54:12.336964 (+ 41us) write_op.cc:276] Timestamp: P: 1777967652336844 usec, L: 0
0505 07:54:12.336969 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.342547 (+ 5578us) log.cc:844] Serialized 765 byte log entry
0505 07:54:12.342812 (+ 265us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65bae8f22 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac
0505 07:54:12.388372 (+ 45560us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.389004 (+ 632us) write_op.cc:301] APPLY: starting
0505 07:54:12.389022 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.389063 (+ 41us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.389067 (+ 4us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.389136 (+ 69us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.389168 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:12.389175 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:12.389236 (+ 61us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.389289 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.389304 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:12.389309 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.389373 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.389377 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":1596800,"child_traces":[["op",{"apply.queue_time_us":576,"apply.run_cpu_time_us":365,"apply.run_wall_time_us":365,"num_ops":2,"prepare.queue_time_us":58,"prepare.run_cpu_time_us":1496,"prepare.run_wall_time_us":6473,"replication_time_us":51384,"spinlock_wait_cycles":12296576}]]}
I20260505 07:54:13.428505 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 571257bbb7e94315b60d84d6aa4917b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:12 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:13.428875 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 571257bbb7e94315b60d84d6aa4917b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:12 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:13.433557 2673 raft_consensus.cc:493] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.433722 2673 raft_consensus.cc:515] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.434372 2673 leader_election.cc:290] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.435325 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b256d072933b4566b5587224274deb66" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.436169 2173 pending_rounds.cc:85] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Aborting all ops after (but not including) 590
I20260505 07:54:13.436230 2414 raft_consensus.cc:515] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.436350 2173 pending_rounds.cc:107] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Aborting uncommitted WRITE_OP operation due to leader change: 1.591
I20260505 07:54:13.436813 2414 leader_election.cc:290] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.436930 2173 pending_rounds.cc:107] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Aborting uncommitted WRITE_OP operation due to leader change: 1.592
I20260505 07:54:13.437183 2173 pending_rounds.cc:107] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Aborting uncommitted WRITE_OP operation due to leader change: 1.593
I20260505 07:54:13.437361 2173 pending_rounds.cc:107] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Aborting uncommitted WRITE_OP operation due to leader change: 1.594
I20260505 07:54:13.437536 2173 pending_rounds.cc:107] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: Aborting uncommitted WRITE_OP operation due to leader change: 1.595
I20260505 07:54:13.438589 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 140) took 3815 ms. Trace:
I20260505 07:54:13.439085 2447 ts_tablet_manager.cc:1929] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.439213 2447 log.cc:1199] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/003cd4472b424aacbba8c5f89900e228
I20260505 07:54:13.442797 2447 ts_tablet_manager.cc:1950] T 003cd4472b424aacbba8c5f89900e228 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:13.442787 2670 raft_consensus.cc:515] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.443722 2670 leader_election.cc:290] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.444473 907 raft_consensus.cc:2468] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:13.446156 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f869abaa31e414ea0625e475db08e64" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:13.446362 918 raft_consensus.cc:3060] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.450888 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e99f96a753047348371bee5c022684e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:13.451129 921 raft_consensus.cc:3060] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:13.438668 2268 rpcz_store.cc:276] 0505 07:54:09.623195 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.623246 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:13.041666 (+3418420us) ts_tablet_manager.cc:1151] Deleting tablet 43e4c92b7c3140fdb2d6a290aaf0ad55
0505 07:54:13.041679 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.149036 (+107357us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.239004 (+ 89968us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.438573 (+199569us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3418296}
I20260505 07:54:13.462513 2268 tablet_replica.cc:333] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:13.462777 2268 raft_consensus.cc:2243] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:13.463202 2268 raft_consensus.cc:2272] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.463886 2268 ts_tablet_manager.cc:1916] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.467629 1047 raft_consensus.cc:2468] T aa049ad4edbd466a860428c822a52724 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:13.469146 991 leader_election.cc:304] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:13.469702 2778 raft_consensus.cc:2804] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:13.469821 2778 raft_consensus.cc:697] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:13.470084 2778 consensus_queue.cc:237] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.471088 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 (table default.Table_43b383b7a58b493c8f5b2d9f3038a2e2 [id=5154509860404937ba84ef134bb50cef]) successfully deleted
I20260505 07:54:13.476102 907 raft_consensus.cc:3060] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.482507 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 4466d0ed977e4803b3b484155095c271
I20260505 07:54:13.483059 1350 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 166) took 2277 ms. Trace:
I20260505 07:54:13.483146 1350 rpcz_store.cc:276] 0505 07:54:11.205630 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.413364 (+1207734us) service_pool.cc:224] Handling call
0505 07:54:12.413958 (+ 594us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:12.414117 (+ 159us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:12.414210 (+ 93us) catalog_manager.cc:2824] Locking tablets
0505 07:54:12.414360 (+ 150us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:12.429469 (+ 15109us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:12.483671 (+ 54202us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:13.482461 (+998790us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:13.483008 (+ 547us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.415114 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.415221 (+ 107us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.415226 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:12.415230 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:12.415392 (+ 162us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.415403 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.415408 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.415514 (+ 106us) tablet.cc:701] Row locks acquired
0505 07:54:12.415516 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:12.415557 (+ 41us) write_op.cc:270] Start()
0505 07:54:12.415607 (+ 50us) write_op.cc:276] Timestamp: P: 1777967652415467 usec, L: 0
0505 07:54:12.415610 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.415785 (+ 175us) log.cc:844] Serialized 725 byte log entry
0505 07:54:12.424816 (+ 9031us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.428308 (+ 3492us) write_op.cc:301] APPLY: starting
0505 07:54:12.428330 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.428384 (+ 54us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.428429 (+ 45us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.428519 (+ 90us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.428566 (+ 47us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:12.428575 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:12.428672 (+ 97us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.428748 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.428773 (+ 25us) write_op.cc:454] Released schema lock
0505 07:54:12.428782 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.429648 (+ 866us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.429655 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3377,"apply.run_cpu_time_us":565,"apply.run_wall_time_us":1238,"num_ops":2,"prepare.queue_time_us":55,"prepare.run_cpu_time_us":746,"prepare.run_wall_time_us":746,"replication_time_us":9166,"spinlock_wait_cycles":896}]]}
I20260505 07:54:13.483959 1350 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_0c1bd9ecad69401ab663ea12961bd890" } new_table_name: "default.Table_001ba75f284249c3b0b479b5b7e27f4c" modify_external_catalogs: true
I20260505 07:54:13.484551 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 4466d0ed977e4803b3b484155095c271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:12 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:13.485096 1357 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet dfbe4dff192741099be5ab10a96adeb7
I20260505 07:54:13.485579 1357 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 176) took 1257 ms. Trace:
I20260505 07:54:13.485915 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 4466d0ed977e4803b3b484155095c271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:12 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:13.485687 1357 rpcz_store.cc:276] 0505 07:54:12.228028 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.780529 (+552501us) service_pool.cc:224] Handling call
0505 07:54:12.781118 (+ 589us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:12.781265 (+ 147us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:12.781319 (+ 54us) catalog_manager.cc:2824] Locking tablets
0505 07:54:12.781470 (+ 151us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:12.803538 (+ 22068us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.803544 (+ 6us) write_op.cc:454] Released schema lock
0505 07:54:12.803640 (+ 96us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:12.803712 (+ 72us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:13.485026 (+681314us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:13.485547 (+ 521us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.782249 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.782365 (+ 116us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.782373 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:12.782376 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:12.782477 (+ 101us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.782487 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.782492 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:12.782588 (+ 96us) tablet.cc:701] Row locks acquired
0505 07:54:12.782591 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:12.782621 (+ 30us) write_op.cc:270] Start()
0505 07:54:12.782653 (+ 32us) write_op.cc:276] Timestamp: P: 1777967652782599 usec, L: 0
0505 07:54:12.782658 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.782827 (+ 169us) log.cc:844] Serialized 725 byte log entry
0505 07:54:12.787292 (+ 4465us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.787379 (+ 87us) write_op.cc:301] APPLY: starting
0505 07:54:12.787391 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.787429 (+ 38us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.787450 (+ 21us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.787521 (+ 71us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.787559 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:12.787564 (+ 5us) write_op.cc:312] APPLY: finished
0505 07:54:12.787639 (+ 75us) log.cc:844] Serialized 29 byte log entry
0505 07:54:12.787691 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.787709 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:12.787714 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":47,"apply.run_cpu_time_us":379,"apply.run_wall_time_us":378,"num_ops":2,"prepare.queue_time_us":115,"prepare.run_cpu_time_us":674,"prepare.run_wall_time_us":674,"raft.queue_time_us":1297,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":227,"replication_time_us":4610}]]}
I20260505 07:54:13.486388 1357 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_a86abb495238445381ac0202804b4ca0" } modify_external_catalogs: true
I20260505 07:54:13.486410 2673 raft_consensus.cc:493] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.486567 2673 raft_consensus.cc:515] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.487175 2673 leader_election.cc:290] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.488629 757 tablet_service.cc:1558] Processing DeleteTablet for tablet dfbe4dff192741099be5ab10a96adeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:12 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:13.490237 887 tablet_service.cc:1558] Processing DeleteTablet for tablet dfbe4dff192741099be5ab10a96adeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:12 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:13.491050 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c837877946604ef29f95dac9dd449bf3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.491262 912 raft_consensus.cc:2393] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
W20260505 07:54:13.491942 718 consensus_peers.cc:597] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9959be9f83c2444282d012b61eef49b5. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:13.500614 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 4466d0ed977e4803b3b484155095c271 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:12 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:13.501169 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet dfbe4dff192741099be5ab10a96adeb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:12 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:13.518715 1176 tablet_bootstrap.cc:492] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.519232 1176 ts_tablet_manager.cc:1403] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.121s user 0.004s sys 0.000s
I20260505 07:54:13.521432 1176 raft_consensus.cc:359] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.521924 1176 raft_consensus.cc:385] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.522119 1176 raft_consensus.cc:740] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.522558 1176 consensus_queue.cc:260] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.523705 1176 ts_tablet_manager.cc:1434] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260505 07:54:13.524490 1176 tablet_bootstrap.cc:492] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.524787 2749 raft_consensus.cc:493] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.524986 2749 raft_consensus.cc:515] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.525723 2749 leader_election.cc:290] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.525897 1172 tablet_bootstrap.cc:492] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:13.526008 1172 ts_tablet_manager.cc:1403] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.342s user 0.021s sys 0.031s
I20260505 07:54:13.526566 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3be796a788e4465a36bb5be44918be1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.526674 1172 raft_consensus.cc:359] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.526824 1172 raft_consensus.cc:385] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.526873 1172 raft_consensus.cc:740] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.527062 1172 consensus_queue.cc:260] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.527487 1172 ts_tablet_manager.cc:1434] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260505 07:54:13.528553 856 leader_election.cc:343] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.532513 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3be796a788e4465a36bb5be44918be1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:13.533329 854 leader_election.cc:343] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.533502 854 leader_election.cc:304] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.533849 2564 raft_consensus.cc:2749] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:13.535820 2276 ts_tablet_manager.cc:1929] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.535773 2650 raft_consensus.cc:515] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.535939 2276 log.cc:1199] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/4258df74ea0841449846cdf1f6e00b9b
I20260505 07:54:13.536322 2276 ts_tablet_manager.cc:1950] T 4258df74ea0841449846cdf1f6e00b9b P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:13.536322 2650 leader_election.cc:290] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.536610 772 raft_consensus.cc:2468] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:13.536633 1878 raft_consensus.cc:515] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.537245 1878 leader_election.cc:290] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.537889 915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe5f6f0774545b296bf1b03a08e145d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:13.538048 915 raft_consensus.cc:3060] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.538196 1348 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_728dc87e78954a7fbbf495d74705e382 [id=2c06ff8eed31432e87f08e9b59049652] alter complete (version 2)
I20260505 07:54:13.540299 921 raft_consensus.cc:2468] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 2.
I20260505 07:54:13.540400 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 157) took 4049 ms. Trace:
I20260505 07:54:13.540601 912 raft_consensus.cc:1240] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Rejecting Update request from peer d63a3e535f8e4dd4a88e447253405b77 for earlier term 1. Current term is 2. Ops: []
I20260505 07:54:13.540520 2447 rpcz_store.cc:276] 0505 07:54:09.491391 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.491436 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:13.347925 (+3856489us) ts_tablet_manager.cc:1151] Deleting tablet 003cd4472b424aacbba8c5f89900e228
0505 07:54:13.347935 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.436031 (+ 88096us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.438995 (+ 2964us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.540360 (+101365us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3856230}
I20260505 07:54:13.540927 2447 tablet_replica.cc:333] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:13.541169 2447 raft_consensus.cc:2243] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:13.541505 2447 raft_consensus.cc:2272] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.542079 2447 ts_tablet_manager.cc:1916] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.543494 1172 tablet_bootstrap.cc:492] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:13.545230 1172 tablet_bootstrap.cc:654] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.546240 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 003cd4472b424aacbba8c5f89900e228 (table default.Table_bd283510c1f14a9fb29a9870d33c9971 [id=ae40934b942a4e0e8ac882d9dab6686f]) successfully deleted
I20260505 07:54:13.548616 718 leader_election.cc:304] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.549300 2414 raft_consensus.cc:2804] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:13.549474 2414 raft_consensus.cc:697] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [term 2 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:13.549750 2414 consensus_queue.cc:237] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.552937 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abe5f6f0774545b296bf1b03a08e145d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:13.553170 772 raft_consensus.cc:3060] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:13.553711 988 leader_election.cc:343] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 8b406c4c6ba741bc85eef54c6f576459
I20260505 07:54:13.554819 2650 consensus_queue.cc:1059] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.555223 2742 raft_consensus.cc:3055] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Stepping down as leader of term 1
I20260505 07:54:13.555291 2742 raft_consensus.cc:740] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:13.555421 1172 tablet_bootstrap.cc:492] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:13.555537 1172 ts_tablet_manager.cc:1403] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260505 07:54:13.555567 2742 consensus_queue.cc:260] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.556129 2742 raft_consensus.cc:3060] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:13.556434 1172 raft_consensus.cc:359] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.556602 1172 raft_consensus.cc:385] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.556650 1172 raft_consensus.cc:740] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.556834 1172 consensus_queue.cc:260] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.557371 1172 ts_tablet_manager.cc:1434] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20260505 07:54:13.561801 991 leader_election.cc:343] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8b406c4c6ba741bc85eef54c6f576459
I20260505 07:54:13.562041 991 leader_election.cc:304] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.562678 1878 raft_consensus.cc:2749] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260505 07:54:13.564313 1878 consensus_queue.cc:1048] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.566905 2564 raft_consensus.cc:493] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.567093 2564 raft_consensus.cc:515] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.567739 2564 leader_election.cc:290] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.568588 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472441bd29704be59aabb0f7a4964bb0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.568905 781 raft_consensus.cc:2468] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:13.569512 854 leader_election.cc:304] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.569939 2564 raft_consensus.cc:2804] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.570035 2564 raft_consensus.cc:493] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.570124 2564 raft_consensus.cc:3060] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:13.576818 856 leader_election.cc:343] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 472441bd29704be59aabb0f7a4964bb0
I20260505 07:54:13.590569 1176 tablet_bootstrap.cc:654] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:13.592041 2772 log.cc:927] Time spent T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77: Append to log took a long time: real 0.075s user 0.000s sys 0.000s
I20260505 07:54:13.599074 1172 tablet_bootstrap.cc:492] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:13.605077 1181 tablet_bootstrap.cc:654] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.615056 2779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23c57a2c927c4c54bccecfeb3a775311. Latest consensus state: current_term: 2 leader_uuid: "23c57a2c927c4c54bccecfeb3a775311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:54:13.615343 2779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:54:13.617199 1878 consensus_queue.cc:1048] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.618232 2806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "23c57a2c927c4c54bccecfeb3a775311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:54:13.618978 2806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260505 07:54:13.630643 1176 tablet_bootstrap.cc:492] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.631249 1176 ts_tablet_manager.cc:1403] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.107s user 0.007s sys 0.000s
I20260505 07:54:13.633778 2447 ts_tablet_manager.cc:1929] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.634469 2173 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36750 (request call id 0) took 6666 ms. Trace:
I20260505 07:54:13.634419 2447 log.cc:1199] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/bd581a6799a14a8dafb80d411e8936f3
I20260505 07:54:13.634778 2268 ts_tablet_manager.cc:1929] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.635003 2268 log.cc:1199] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/4783dd0b2fbe40219852d850fc0d9a82
I20260505 07:54:13.635527 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 107) took 5854 ms. Trace:
I20260505 07:54:13.635675 2276 rpcz_store.cc:276] 0505 07:54:07.781377 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.798727 (+ 17350us) service_pool.cc:224] Handling call
0505 07:54:13.399820 (+5601093us) ts_tablet_manager.cc:1151] Deleting tablet 4258df74ea0841449846cdf1f6e00b9b
0505 07:54:13.399831 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.439734 (+ 39903us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.535757 (+ 96023us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.635473 (+ 99716us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5600924}
I20260505 07:54:13.636093 2276 tablet_replica.cc:333] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:13.636314 2276 raft_consensus.cc:2243] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:13.636547 2276 raft_consensus.cc:2272] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.637144 2276 ts_tablet_manager.cc:1916] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.637784 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 4258df74ea0841449846cdf1f6e00b9b (table default.Table_b3b9364eea524724b176de87f62eb49e [id=1fd77c8cfe9d40dab5da490774294796]) successfully deleted
I20260505 07:54:13.634126 1176 raft_consensus.cc:359] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.638325 1176 raft_consensus.cc:385] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.638893 2268 ts_tablet_manager.cc:1950] T 4783dd0b2fbe40219852d850fc0d9a82 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:13.638989 1176 raft_consensus.cc:740] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.641191 2447 ts_tablet_manager.cc:1950] T bd581a6799a14a8dafb80d411e8936f3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:13.642026 2777 mvcc.cc:204] Tried to move back new op lower bound from 7282555481373491200 to 7282555461753741312. Current Snapshot: MvccSnapshot[applied={T|T < 7282555481368391680}]
I20260505 07:54:13.641757 1172 tablet_bootstrap.cc:654] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.642588 2813 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260505 07:54:13.645491 918 raft_consensus.cc:2468] T 3f869abaa31e414ea0625e475db08e64 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:13.652024 718 leader_election.cc:304] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.652520 2670 raft_consensus.cc:2804] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:13.652875 2670 raft_consensus.cc:697] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:13.653250 2670 consensus_queue.cc:237] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.658684 1349 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 161) took 2546 ms. Trace:
I20260505 07:54:13.658797 1349 rpcz_store.cc:276] 0505 07:54:11.112551 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.061213 (+948662us) service_pool.cc:224] Handling call
0505 07:54:12.244523 (+183310us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:12.244738 (+ 215us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:12.244903 (+ 165us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:12.244959 (+ 56us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:12.388911 (+143952us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:13.658619 (+1269708us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.247003 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.247088 (+ 85us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.247092 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:12.247094 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:12.247163 (+ 69us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.247171 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.247175 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:12.247215 (+ 40us) tablet.cc:701] Row locks acquired
0505 07:54:12.247217 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:12.247240 (+ 23us) write_op.cc:270] Start()
0505 07:54:12.247260 (+ 20us) write_op.cc:276] Timestamp: P: 1777967652247176 usec, L: 0
0505 07:54:12.247264 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.247378 (+ 114us) log.cc:844] Serialized 405 byte log entry
0505 07:54:12.388238 (+140860us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.388465 (+ 227us) write_op.cc:301] APPLY: starting
0505 07:54:12.388480 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.388528 (+ 48us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.388561 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.388612 (+ 51us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.388647 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:12.388654 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:12.388733 (+ 79us) log.cc:844] Serialized 23 byte log entry
0505 07:54:12.388805 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.388821 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:12.388832 (+ 11us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.388913 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.388917 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":153,"apply.run_cpu_time_us":443,"apply.run_wall_time_us":443,"num_ops":1,"prepare.queue_time_us":1704,"prepare.run_cpu_time_us":432,"prepare.run_wall_time_us":432,"replication_time_us":140940}]]}
I20260505 07:54:13.659566 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_d0e52326b0e041e3941fd45dab79f2da" } new_table_name: "default.Table_94a492cf2db14ff3900cf09b17058af3" modify_external_catalogs: true
I20260505 07:54:13.661530 1357 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_a86abb495238445381ac0202804b4ca0" } modify_external_catalogs: true
I20260505 07:54:13.673115 1176 consensus_queue.cc:260] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.673264 2674 raft_consensus.cc:515] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.675930 1176 ts_tablet_manager.cc:1434] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.042s user 0.003s sys 0.000s
I20260505 07:54:13.675985 2674 leader_election.cc:290] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.676015 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a0cbf9c3d74cea92920189da332c1c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:13.676211 918 raft_consensus.cc:3060] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.676241 1176 tablet_bootstrap.cc:492] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.678421 1181 tablet_bootstrap.cc:492] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:13.678689 1181 ts_tablet_manager.cc:1403] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.260s user 0.000s sys 0.005s
I20260505 07:54:13.679883 1181 raft_consensus.cc:359] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.680207 1181 raft_consensus.cc:385] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.680337 1181 raft_consensus.cc:740] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.680624 1181 consensus_queue.cc:260] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.685248 1172 tablet_bootstrap.cc:492] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:13.685395 1172 ts_tablet_manager.cc:1403] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.088s user 0.003s sys 0.003s
I20260505 07:54:13.686205 1172 raft_consensus.cc:359] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.686388 1172 raft_consensus.cc:385] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.686452 1172 raft_consensus.cc:740] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.686645 1172 consensus_queue.cc:260] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.687158 1172 ts_tablet_manager.cc:1434] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:13.687438 1172 tablet_bootstrap.cc:492] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:13.693876 1172 tablet_bootstrap.cc:654] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:13.698449 991 consensus_peers.cc:597] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f1058fcb9fa6457a85489049ca1d9ada. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:13.700431 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 141) took 4076 ms. Trace:
I20260505 07:54:13.700542 2268 rpcz_store.cc:276] 0505 07:54:09.623621 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.623664 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:13.462446 (+3838782us) ts_tablet_manager.cc:1151] Deleting tablet 4783dd0b2fbe40219852d850fc0d9a82
0505 07:54:13.462463 (+ 17us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.535761 (+ 73298us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.634709 (+ 98948us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.700407 (+ 65698us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3838655}
I20260505 07:54:13.701097 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 4783dd0b2fbe40219852d850fc0d9a82 (table default.Table_0555c54d8e394cb1a8509950422c9ac8 [id=e59beab126684f548325a29eb663177b]) successfully deleted
I20260505 07:54:13.701517 2268 tablet_replica.cc:333] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:13.701786 2268 raft_consensus.cc:2243] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:13.701947 2268 raft_consensus.cc:2272] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.702684 2268 ts_tablet_manager.cc:1916] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.705830 2136 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" instance_seqno: 1777967632532909) as {username='slave'} at 127.0.105.3:34327; Asking this server to re-register.
I20260505 07:54:13.711719 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 158) took 4206 ms. Trace:
I20260505 07:54:13.711843 2447 rpcz_store.cc:276] 0505 07:54:09.491839 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.491904 (+ 65us) service_pool.cc:224] Handling call
0505 07:54:13.540826 (+4048922us) ts_tablet_manager.cc:1151] Deleting tablet bd581a6799a14a8dafb80d411e8936f3
0505 07:54:13.540857 (+ 31us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.630587 (+ 89730us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.633697 (+ 3110us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.698095 (+ 64398us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4048685}
I20260505 07:54:13.712267 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 159) took 4219 ms. Trace:
I20260505 07:54:13.712369 2447 rpcz_store.cc:276] 0505 07:54:09.492281 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.492326 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:13.712125 (+4219799us) ts_tablet_manager.cc:1151] Deleting tablet 840eebf9e7404bf7807b7e51996957e5
0505 07:54:13.712135 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.712214 (+ 79us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4219527}
I20260505 07:54:13.712657 2447 tablet_replica.cc:333] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:13.712858 2447 raft_consensus.cc:2243] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:13.713193 2447 raft_consensus.cc:2272] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.713759 2447 ts_tablet_manager.cc:1916] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.715991 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet bd581a6799a14a8dafb80d411e8936f3 (table default.Table_822258eaa3b14528a8bfa97e3e9d4d7a [id=76b0eb99a1f343829e5ba35de6302cf2]) successfully deleted
W20260505 07:54:13.716292 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 840eebf9e7404bf7807b7e51996957e5 because the tablet was not found. No further retry: Not found: Tablet not found: 840eebf9e7404bf7807b7e51996957e5
I20260505 07:54:13.716531 2749 raft_consensus.cc:493] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.716671 2749 raft_consensus.cc:515] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.717242 2749 leader_election.cc:290] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:13.717783 718 consensus_peers.cc:597] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 141665fcd59048bf926acd0287f1853e. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:13.717770 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4515a0ff8304c69b9aa030e9b14d43f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:13.718236 856 leader_election.cc:343] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.718287 2749 raft_consensus.cc:493] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.718446 2749 raft_consensus.cc:515] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.718883 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4515a0ff8304c69b9aa030e9b14d43f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.718977 2749 leader_election.cc:290] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.719524 2742 raft_consensus.cc:493] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.719622 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02716f62aa354d8f839266661b7f05ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.719715 2742 raft_consensus.cc:515] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.720224 2742 leader_election.cc:290] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:13.720418 856 leader_election.cc:343] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:13.720749 854 leader_election.cc:343] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.720851 854 leader_election.cc:304] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.721536 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02716f62aa354d8f839266661b7f05ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.721944 2749 raft_consensus.cc:2749] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:13.722190 854 leader_election.cc:343] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.722245 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7d8265c05b74b28959d0de41fad1fde" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.722301 854 leader_election.cc:304] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.722419 776 raft_consensus.cc:2468] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:13.722651 2749 raft_consensus.cc:2749] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:13.723093 991 leader_election.cc:343] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: a7d8265c05b74b28959d0de41fad1fde
I20260505 07:54:13.723937 2650 consensus_queue.cc:1048] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:13.724368 718 consensus_peers.cc:597] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 03f31efbc98c484c950c70d5e84d2136. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:13.724371 2650 consensus_queue.cc:1048] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.725001 988 leader_election.cc:304] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:13.725414 1878 raft_consensus.cc:2804] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.725492 1878 raft_consensus.cc:493] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.725569 1878 raft_consensus.cc:3060] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.725963 2673 raft_consensus.cc:493] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.726596 915 raft_consensus.cc:2468] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:13.728986 2564 raft_consensus.cc:515] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.729508 2564 leader_election.cc:290] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.729964 991 leader_election.cc:304] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:13.730337 2650 raft_consensus.cc:2804] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:13.730424 2650 raft_consensus.cc:697] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:13.730593 2650 consensus_queue.cc:237] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.636354 2173 rpcz_store.cc:276] 0505 07:54:06.968333 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:06.968384 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:13.338833 (+6370449us) raft_consensus.cc:1471] Updating replica for 60 ops
0505 07:54:13.436764 (+ 97931us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.436768 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.437098 (+ 330us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.437101 (+ 3us) write_op.cc:454] Released schema lock
0505 07:54:13.437307 (+ 206us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.437310 (+ 3us) write_op.cc:454] Released schema lock
0505 07:54:13.437469 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.437472 (+ 3us) write_op.cc:454] Released schema lock
0505 07:54:13.437665 (+ 193us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.437668 (+ 3us) write_op.cc:454] Released schema lock
0505 07:54:13.438072 (+ 404us) raft_consensus.cc:1534] Early marking committed up to index 590
0505 07:54:13.466831 (+ 28759us) raft_consensus.cc:1539] Triggering prepare for 60 ops
0505 07:54:13.467242 (+ 411us) write_op.cc:270] Start()
0505 07:54:13.467324 (+ 82us) write_op.cc:276] Timestamp: P: 1777967645795474 usec, L: 0
0505 07:54:13.467615 (+ 291us) write_op.cc:270] Start()
0505 07:54:13.467644 (+ 29us) write_op.cc:276] Timestamp: P: 1777967645820192 usec, L: 0
0505 07:54:13.467842 (+ 198us) write_op.cc:270] Start()
0505 07:54:13.467864 (+ 22us) write_op.cc:276] Timestamp: P: 1777967645820806 usec, L: 0
0505 07:54:13.467991 (+ 127us) write_op.cc:270] Start()
0505 07:54:13.468012 (+ 21us) write_op.cc:276] Timestamp: P: 1777967645821451 usec, L: 0
0505 07:54:13.468184 (+ 172us) write_op.cc:270] Start()
0505 07:54:13.468204 (+ 20us) write_op.cc:276] Timestamp: P: 1777967645822093 usec, L: 0
0505 07:54:13.468362 (+ 158us) write_op.cc:270] Start()
0505 07:54:13.468388 (+ 26us) write_op.cc:276] Timestamp: P: 1777967645822697 usec, L: 0
0505 07:54:13.468525 (+ 137us) write_op.cc:270] Start()
0505 07:54:13.468563 (+ 38us) write_op.cc:276] Timestamp: P: 1777967645823330 usec, L: 0
0505 07:54:13.468718 (+ 155us) write_op.cc:270] Start()
0505 07:54:13.468742 (+ 24us) write_op.cc:276] Timestamp: P: 1777967645828276 usec, L: 0
0505 07:54:13.468881 (+ 139us) write_op.cc:270] Start()
0505 07:54:13.468900 (+ 19us) write_op.cc:276] Timestamp: P: 1777967645828969 usec, L: 0
0505 07:54:13.469036 (+ 136us) write_op.cc:270] Start()
0505 07:54:13.469054 (+ 18us) write_op.cc:276] Timestamp: P: 1777967645908128 usec, L: 0
0505 07:54:13.481647 (+ 12593us) write_op.cc:270] Start()
0505 07:54:13.481687 (+ 40us) write_op.cc:276] Timestamp: P: 1777967645909850 usec, L: 0
0505 07:54:13.485164 (+ 3477us) write_op.cc:270] Start()
0505 07:54:13.485191 (+ 27us) write_op.cc:276] Timestamp: P: 1777967645917725 usec, L: 0
0505 07:54:13.485395 (+ 204us) write_op.cc:270] Start()
0505 07:54:13.485419 (+ 24us) write_op.cc:276] Timestamp: P: 1777967645945881 usec, L: 0
0505 07:54:13.485575 (+ 156us) write_op.cc:270] Start()
0505 07:54:13.485600 (+ 25us) write_op.cc:276] Timestamp: P: 1777967645952313 usec, L: 0
0505 07:54:13.485773 (+ 173us) write_op.cc:270] Start()
0505 07:54:13.485793 (+ 20us) write_op.cc:276] Timestamp: P: 1777967645953922 usec, L: 0
0505 07:54:13.485914 (+ 121us) write_op.cc:270] Start()
0505 07:54:13.485940 (+ 26us) write_op.cc:276] Timestamp: P: 1777967646002951 usec, L: 0
0505 07:54:13.486068 (+ 128us) write_op.cc:270] Start()
0505 07:54:13.486086 (+ 18us) write_op.cc:276] Timestamp: P: 1777967646011829 usec, L: 0
0505 07:54:13.486205 (+ 119us) write_op.cc:270] Start()
0505 07:54:13.486225 (+ 20us) write_op.cc:276] Timestamp: P: 1777967646128383 usec, L: 0
0505 07:54:13.486427 (+ 202us) write_op.cc:270] Start()
0505 07:54:13.486448 (+ 21us) write_op.cc:276] Timestamp: P: 1777967646135860 usec, L: 0
0505 07:54:13.486612 (+ 164us) write_op.cc:270] Start()
0505 07:54:13.486635 (+ 23us) write_op.cc:276] Timestamp: P: 1777967646149533 usec, L: 0
0505 07:54:13.486815 (+ 180us) write_op.cc:270] Start()
0505 07:54:13.486846 (+ 31us) write_op.cc:276] Timestamp: P: 1777967646162299 usec, L: 0
0505 07:54:13.487002 (+ 156us) write_op.cc:270] Start()
0505 07:54:13.487022 (+ 20us) write_op.cc:276] Timestamp: P: 1777967646173005 usec, L: 0
0505 07:54:13.487159 (+ 137us) write_op.cc:270] Start()
0505 07:54:13.487181 (+ 22us) write_op.cc:276] Timestamp: P: 1777967646186421 usec, L: 0
0505 07:54:13.487313 (+ 132us) write_op.cc:270] Start()
0505 07:54:13.487333 (+ 20us) write_op.cc:276] Timestamp: P: 1777967646263011 usec, L: 0
0505 07:54:13.487515 (+ 182us) write_op.cc:270] Start()
0505 07:54:13.487537 (+ 22us) write_op.cc:276] Timestamp: P: 1777967646288726 usec, L: 0
0505 07:54:13.505016 (+ 17479us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x616000030e30. stack: 00007fc574ffb086 00007fc574ffae40 00007fc573fb43ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fc5822a17af 00007fc580f02c81 00007fc580f223d7 00007fc580f1fcc7 00007fc584da459d 00007fc57ef30b46 00007fc57ef30875 00007fc57bfee10e 00007fc57bfecfee 00007fc57bff3228
0505 07:54:13.505126 (+ 110us) write_op.cc:270] Start()
0505 07:54:13.505157 (+ 31us) write_op.cc:276] Timestamp: P: 1777967646396940 usec, L: 0
0505 07:54:13.505338 (+ 181us) write_op.cc:270] Start()
0505 07:54:13.505361 (+ 23us) write_op.cc:276] Timestamp: P: 1777967646407158 usec, L: 0
0505 07:54:13.505497 (+ 136us) write_op.cc:270] Start()
0505 07:54:13.505517 (+ 20us) write_op.cc:276] Timestamp: P: 1777967646421834 usec, L: 0
0505 07:54:13.505641 (+ 124us) write_op.cc:270] Start()
0505 07:54:13.505690 (+ 49us) write_op.cc:276] Timestamp: P: 1777967646438429 usec, L: 0
0505 07:54:13.505842 (+ 152us) write_op.cc:270] Start()
0505 07:54:13.505859 (+ 17us) write_op.cc:276] Timestamp: P: 1777967646448760 usec, L: 0
0505 07:54:13.505978 (+ 119us) write_op.cc:270] Start()
0505 07:54:13.505996 (+ 18us) write_op.cc:276] Timestamp: P: 1777967646472128 usec, L: 0
0505 07:54:13.506137 (+ 141us) write_op.cc:270] Start()
0505 07:54:13.506158 (+ 21us) write_op.cc:276] Timestamp: P: 1777967646475492 usec, L: 0
0505 07:54:13.506296 (+ 138us) write_op.cc:270] Start()
0505 07:54:13.506315 (+ 19us) write_op.cc:276] Timestamp: P: 1777967646478705 usec, L: 0
0505 07:54:13.506428 (+ 113us) write_op.cc:270] Start()
0505 07:54:13.506448 (+ 20us) write_op.cc:276] Timestamp: P: 1777967646483165 usec, L: 0
0505 07:54:13.506582 (+ 134us) write_op.cc:270] Start()
0505 07:54:13.506600 (+ 18us) write_op.cc:276] Timestamp: P: 1777967646486245 usec, L: 0
0505 07:54:13.506744 (+ 144us) write_op.cc:270] Start()
0505 07:54:13.506763 (+ 19us) write_op.cc:276] Timestamp: P: 1777967646500326 usec, L: 0
0505 07:54:13.530008 (+ 23245us) write_op.cc:270] Start()
0505 07:54:13.530055 (+ 47us) write_op.cc:276] Timestamp: P: 1777967646501328 usec, L: 0
0505 07:54:13.530266 (+ 211us) write_op.cc:270] Start()
0505 07:54:13.530291 (+ 25us) write_op.cc:276] Timestamp: P: 1777967646503608 usec, L: 0
0505 07:54:13.530421 (+ 130us) write_op.cc:270] Start()
0505 07:54:13.530437 (+ 16us) write_op.cc:276] Timestamp: P: 1777967646618093 usec, L: 0
0505 07:54:13.530553 (+ 116us) write_op.cc:270] Start()
0505 07:54:13.530569 (+ 16us) write_op.cc:276] Timestamp: P: 1777967646658192 usec, L: 0
0505 07:54:13.530701 (+ 132us) write_op.cc:270] Start()
0505 07:54:13.530718 (+ 17us) write_op.cc:276] Timestamp: P: 1777967646660418 usec, L: 0
0505 07:54:13.530829 (+ 111us) write_op.cc:270] Start()
0505 07:54:13.530844 (+ 15us) write_op.cc:276] Timestamp: P: 1777967646675221 usec, L: 0
0505 07:54:13.530961 (+ 117us) write_op.cc:270] Start()
0505 07:54:13.530979 (+ 18us) write_op.cc:276] Timestamp: P: 1777967646676442 usec, L: 0
0505 07:54:13.531101 (+ 122us) write_op.cc:270] Start()
0505 07:54:13.531124 (+ 23us) write_op.cc:276] Timestamp: P: 1777967646693071 usec, L: 0
0505 07:54:13.531330 (+ 206us) write_op.cc:270] Start()
0505 07:54:13.531350 (+ 20us) write_op.cc:276] Timestamp: P: 1777967646694016 usec, L: 0
0505 07:54:13.531508 (+ 158us) write_op.cc:270] Start()
0505 07:54:13.531529 (+ 21us) write_op.cc:276] Timestamp: P: 1777967646712514 usec, L: 0
0505 07:54:13.531672 (+ 143us) write_op.cc:270] Start()
0505 07:54:13.531695 (+ 23us) write_op.cc:276] Timestamp: P: 1777967646725649 usec, L: 0
0505 07:54:13.531842 (+ 147us) write_op.cc:270] Start()
0505 07:54:13.531867 (+ 25us) write_op.cc:276] Timestamp: P: 1777967646745580 usec, L: 0
0505 07:54:13.532013 (+ 146us) write_op.cc:270] Start()
0505 07:54:13.532031 (+ 18us) write_op.cc:276] Timestamp: P: 1777967646753733 usec, L: 0
0505 07:54:13.532146 (+ 115us) write_op.cc:270] Start()
0505 07:54:13.532163 (+ 17us) write_op.cc:276] Timestamp: P: 1777967646762119 usec, L: 0
0505 07:54:13.532324 (+ 161us) write_op.cc:270] Start()
0505 07:54:13.532362 (+ 38us) write_op.cc:276] Timestamp: P: 1777967646793091 usec, L: 0
0505 07:54:13.532474 (+ 112us) write_op.cc:270] Start()
0505 07:54:13.532494 (+ 20us) write_op.cc:276] Timestamp: P: 1777967646795077 usec, L: 0
0505 07:54:13.532633 (+ 139us) write_op.cc:270] Start()
0505 07:54:13.532662 (+ 29us) write_op.cc:276] Timestamp: P: 1777967646797318 usec, L: 0
0505 07:54:13.532811 (+ 149us) write_op.cc:270] Start()
0505 07:54:13.532835 (+ 24us) write_op.cc:276] Timestamp: P: 1777967646802972 usec, L: 0
0505 07:54:13.532992 (+ 157us) write_op.cc:270] Start()
0505 07:54:13.533013 (+ 21us) write_op.cc:276] Timestamp: P: 1777967646808446 usec, L: 0
0505 07:54:13.533131 (+ 118us) write_op.cc:270] Start()
0505 07:54:13.533160 (+ 29us) write_op.cc:276] Timestamp: P: 1777967646809237 usec, L: 0
0505 07:54:13.533278 (+ 118us) write_op.cc:270] Start()
0505 07:54:13.533300 (+ 22us) write_op.cc:276] Timestamp: P: 1777967646809979 usec, L: 0
0505 07:54:13.533435 (+ 135us) write_op.cc:270] Start()
0505 07:54:13.533454 (+ 19us) write_op.cc:276] Timestamp: P: 1777967646818455 usec, L: 0
0505 07:54:13.533587 (+ 133us) write_op.cc:270] Start()
0505 07:54:13.533605 (+ 18us) write_op.cc:276] Timestamp: P: 1777967646819700 usec, L: 0
0505 07:54:13.535249 (+ 1644us) log.cc:844] Serialized 43311 byte log entry
0505 07:54:13.553288 (+ 18039us) raft_consensus.cc:1645] Marking committed up to 650
0505 07:54:13.554636 (+ 1348us) log.cc:844] Serialized 15 byte log entry
0505 07:54:13.563070 (+ 8434us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.563077 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:13.564257 (+ 1180us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.564262 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:13.565052 (+ 790us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.565058 (+ 6us) write_op.cc:454] Released schema lock
0505 07:54:13.565802 (+ 744us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.565806 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.566611 (+ 805us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.566615 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.567439 (+ 824us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.567444 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:13.568344 (+ 900us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.568350 (+ 6us) write_op.cc:454] Released schema lock
0505 07:54:13.569054 (+ 704us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.569058 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.570013 (+ 955us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.570017 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.570732 (+ 715us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.570737 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:13.571496 (+ 759us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.571500 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.572275 (+ 775us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.572280 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:13.572992 (+ 712us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.572999 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:13.573956 (+ 957us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.573963 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:13.574899 (+ 936us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.574903 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.575693 (+ 790us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.575697 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.576479 (+ 782us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.576483 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.577452 (+ 969us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.577458 (+ 6us) write_op.cc:454] Released schema lock
0505 07:54:13.578251 (+ 793us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.578255 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.579030 (+ 775us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.579035 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:13.579839 (+ 804us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.579843 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.580623 (+ 780us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.580627 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.581500 (+ 873us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.581507 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:13.582840 (+ 1333us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.582846 (+ 6us) write_op.cc:454] Released schema lock
0505 07:54:13.584210 (+ 1364us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.584215 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:13.584953 (+ 738us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.584960 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:13.585862 (+ 902us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.585866 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.590569 (+ 4703us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.590574 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:13.592165 (+ 1591us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.592169 (+ 4us) write_op.cc:454] Released schema lock
0505 07:54:13.596809 (+ 4640us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.596816 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:13.601530 (+ 4714us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.601537 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:13.602446 (+ 909us) raft_consensus.cc:1697] Filling consensus response to leader.
0505 07:54:13.614983 (+ 12537us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x616000031054. stack: 00007fc574ffb086 00007fc574ffae40 00007fc573fb43ad 000000000099adb4 000000000099ad7d 00007fc5843836bd 00007fc5843837b3 00007fc580f41b21 00007fc588e65665 00007fc588e794c5 00007fc588e79271 00007fc5822c2c9d 00007fc580f5292e 00007fc580f5267d 0000000000aa4cac 00007fc575040f6f
0505 07:54:13.618183 (+ 3200us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x616000031054. stack: 00007fc574ffb086 00007fc574ffae40 00007fc573fb43ad 000000000099adb4 000000000099ad7d 00007fc5843836bd 00007fc5843837b3 00007fc580f41b21 00007fc588e65665 00007fc588e794c5 00007fc588e79271 00007fc5822c2c9d 00007fc580f5292e 00007fc580f5267d 0000000000aa4cac 00007fc575040f6f
0505 07:54:13.634211 (+ 16028us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x616000031054. stack: 00007fc574ffb086 00007fc574ffae40 00007fc573fb43ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fc580f23f3b 00007fc580f1fcc7 00007fc584da459d 00007fc57ef30b46 00007fc57ef30875 00007fc57bfee10e 00007fc57bfecfee 00007fc57bff3228 00007fc57bff3b8f 00007fc57bff39b8
0505 07:54:13.634222 (+ 11us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0505 07:54:13.634258 (+ 36us) raft_consensus.cc:1685] finished
0505 07:54:13.634265 (+ 7us) raft_consensus.cc:1691] UpdateReplicas() finished
0505 07:54:13.634431 (+ 166us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:13.467476 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.467688 (+ 212us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.467695 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:13.467698 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:13.467823 (+ 125us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.467862 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.467869 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:13.467960 (+ 91us) tablet.cc:701] Row locks acquired
0505 07:54:13.467963 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:13.561930 (+ 93967us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.562136 (+ 206us) write_op.cc:301] APPLY: starting
0505 07:54:13.562158 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.562229 (+ 71us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.562233 (+ 4us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.562354 (+ 121us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.562431 (+ 77us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:13.562464 (+ 33us) write_op.cc:312] APPLY: finished
0505 07:54:13.562557 (+ 93us) log.cc:844] Serialized 29 byte log entry
0505 07:54:13.562633 (+ 76us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.562678 (+ 45us) write_op.cc:454] Released schema lock
0505 07:54:13.562686 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:13.468032 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.468158 (+ 126us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.468163 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:13.468165 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:13.468286 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.468300 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.468305 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:13.468404 (+ 99us) tablet.cc:701] Row locks acquired
0505 07:54:13.468406 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:13.563167 (+ 94761us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.563681 (+ 514us) write_op.cc:301] APPLY: starting
0505 07:54:13.563696 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.563730 (+ 34us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.563732 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.563789 (+ 57us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.563824 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:13.563855 (+ 31us) write_op.cc:312] APPLY: finished
0505 07:54:13.563914 (+ 59us) log.cc:844] Serialized 29 byte log entry
0505 07:54:13.563974 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.563989 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:13.563994 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:13.468475 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.468609 (+ 134us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.468615 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:13.468617 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:13.468693 (+ 76us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.468703 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.468709 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:13.468762 (+ 53us) tablet.cc:701] Row locks acquired
0505 07:54:13.468764 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:13.564331 (+ 95567us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.564551 (+ 220us) write_op.cc:301] APPLY: starting
0505 07:54:13.564565 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.564594 (+ 29us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.564599 (+ 5us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.564629 (+ 30us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.564668 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:13.564705 (+ 37us) write_op.cc:312] APPLY: finished
0505 07:54:13.564758 (+ 53us) log.cc:844] Serialized 23 byte log entry
0505 07:54:13.564818 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.564829 (+ 11us) write_op.cc:454] Released schema lock
0505 07:54:13.564835 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:13.468847 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.468984 (+ 137us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.468988 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:13.468991 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:13.469065 (+ 74us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.469076 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.469081 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:13.469144 (+ 63us) tablet.cc:701] Row locks acquired
0505 07:54:13.469146 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:13.565126 (+ 95980us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.565322 (+ 196us) write_op.cc:301] APPLY: starting
0505 07:54:13.565335 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.565361 (+ 26us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.565364 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.565397 (+ 33us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.565419 (+ 22us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:13.565451 (+ 32us) write_op.cc:312] APPLY: finished
0505 07:54:13.565500 (+ 49us) log.cc:844] Serialized 23 byte log entry
0505 07:54:13.565551 (+ 51us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.565562 (+ 11us) write_op.cc:454] Released schema lock
0505 07:54:13.565567 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:13.469215 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.469296 (+ 81us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.469301 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:13.469303 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:13.469371 (+ 68us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.469383 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.469390 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:13.469456 (+ 66us) tablet.cc:701] Row locks acquired
0505 07:54:13.469458 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:13.565870 (+ 96412us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.566059 (+ 189us) write_op.cc:301] APPLY: starting
0505 07:54:13.566071 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.566103 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.566105 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.566205 (+ 100us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.566234 (+ 29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:13.566261 (+ 27us) write_op.cc:312] APPLY: finished
0505 07:54:13.566316 (+ 55us) log.cc:844] Serialized 29 byte log entry
0505 07:54:13.566368 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.566396 (+ 28us) write_op.cc:454] Released schema lock
0505 07:54:13.566402 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:13.469525 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.469620 (+ 95us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.469626 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:13.469628 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:13.469699 (+ 71us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.469710 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.469718 (+ 8us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:13.469785 (+ 67us) tablet.cc:701] Row locks acquired
0505 07:54:13.469788 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:13.566690 (+ 96902us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.566896 (+ 206us) write_op.cc:301] APPLY: starting
0505 07:54:13.566907 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.566935 (+ 28us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.566937 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.567030 (+ 93us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.567056 (+ 26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:13.567096 (+ 40us) write_op.cc:312] APPLY: finished
0505 07:54:13.567145 (+ 49us) log.cc:844] Serialized 29 byte log entry
0505 07:54:13.567199 (+ 54us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.567223 (+ 24us) write_op.cc:454] Released schema lock
0505 07:54:13.567228 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:13.469855 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.469951 (+ 96us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.469955 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:13.469958 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:13.470019 (+ 61us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.470029 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.470033 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:13.470090 (+ 57us) tablet.cc:701] Row locks acquired
0505 07:54:13.470093 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:13.567506 (+ 97413us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.567702 (+ 196us) write_op.cc:301] APPLY: starting
0505 07:54:13.567712 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.567759 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.567763 (+ 4us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.567909 (+ 146us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.567932 (+ 23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:13.567960 (+ 28us) write_op.cc:312] APPLY: finished
0505 07:54:13.568013 (+ 53us) log.cc:844] Serialized 29 byte log entry
0505 07:54:13.568090 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.568105 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:13.568113 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:13.470196 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.470308 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.470313 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:13.470315 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:13.470381 (+ 66us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.470391 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.470732 (+ 341us) tablet.cc
I20260505 07:54:13.738343 2778 consensus_queue.cc:1048] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:13.738718 856 leader_election.cc:343] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 472441bd29704be59aabb0f7a4964bb0
I20260505 07:54:13.743191 2302 raft_consensus.cc:493] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.743326 2302 raft_consensus.cc:515] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.743750 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "472441bd29704be59aabb0f7a4964bb0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:13.743891 2302 leader_election.cc:290] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.744369 767 raft_consensus.cc:3060] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.745031 2673 raft_consensus.cc:515] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.745538 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a0cbf9c3d74cea92920189da332c1c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.746342 1176 tablet_bootstrap.cc:654] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.748625 2673 leader_election.cc:290] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.752230 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e99f96a753047348371bee5c022684e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.752445 1047 raft_consensus.cc:2410] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260505 07:54:13.752751 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7d8265c05b74b28959d0de41fad1fde" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.753099 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f106a6d4855148fe83b1e90ea9f27d39" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.753252 1048 raft_consensus.cc:2393] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:13.753893 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a0cbf9c3d74cea92920189da332c1c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.754042 1048 raft_consensus.cc:2468] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:13.754375 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b256d072933b4566b5587224274deb66" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.754595 1048 raft_consensus.cc:2393] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:13.754926 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f869abaa31e414ea0625e475db08e64" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:13.755061 1048 raft_consensus.cc:3060] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.755440 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e99f96a753047348371bee5c022684e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:13.756126 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c837877946604ef29f95dac9dd449bf3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.760370 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a0cbf9c3d74cea92920189da332c1c" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:13.760510 1053 raft_consensus.cc:3060] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.762414 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4dd3ece5cb34ceb82feeb67e75da0b5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.762565 1051 raft_consensus.cc:2468] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:13.763777 717 leader_election.cc:304] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:13.765283 2673 raft_consensus.cc:3060] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.765734 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4dd3ece5cb34ceb82feeb67e75da0b5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.765926 921 raft_consensus.cc:2468] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:13.767330 2280 raft_consensus.cc:493] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.767477 2280 raft_consensus.cc:515] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.768165 2280 leader_election.cc:290] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:13.768646 717 consensus_peers.cc:597] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4258df74ea0841449846cdf1f6e00b9b. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:13.768796 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5a0cbf9c3d74cea92920189da332c1c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.768980 777 raft_consensus.cc:2393] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:13.772621 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3462dcc6391741e59dcb5cd08527e50f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.772725 988 leader_election.cc:304] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.772804 921 raft_consensus.cc:2468] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:13.776394 2136 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" instance_seqno: 1777967631556394) as {username='slave'} at 127.0.105.1:38643; Asking this server to re-register.
I20260505 07:54:13.777592 1181 ts_tablet_manager.cc:1434] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.099s user 0.002s sys 0.000s
I20260505 07:54:13.778406 717 leader_election.cc:304] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:13.778869 2674 raft_consensus.cc:3060] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.780004 1181 tablet_bootstrap.cc:492] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:13.780522 717 leader_election.cc:304] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:13.780776 2280 raft_consensus.cc:3060] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.782411 827 heartbeater.cc:461] Registering TS with master...
I20260505 07:54:13.782754 717 leader_election.cc:304] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.783077 2670 raft_consensus.cc:2804] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.783231 2670 raft_consensus.cc:493] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.783372 2670 raft_consensus.cc:3060] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.784757 1181 tablet_bootstrap.cc:654] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.793154 2672 consensus_queue.cc:1048] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.797122 1176 tablet_bootstrap.cc:492] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.797304 1176 ts_tablet_manager.cc:1403] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.121s user 0.004s sys 0.005s
I20260505 07:54:13.798094 1176 raft_consensus.cc:359] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.798278 1176 raft_consensus.cc:385] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.798360 1176 raft_consensus.cc:740] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.798605 1176 consensus_queue.cc:260] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.799070 1176 ts_tablet_manager.cc:1434] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:13.799351 1176 tablet_bootstrap.cc:492] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.800937 718 leader_election.cc:304] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.801297 1176 tablet_bootstrap.cc:654] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.801426 2672 raft_consensus.cc:2804] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.801559 2672 raft_consensus.cc:493] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.801676 2672 raft_consensus.cc:3060] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.801532 2802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "23c57a2c927c4c54bccecfeb3a775311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:54:13.801767 2802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: This master's current role is: LEADER
I20260505 07:54:13.820363 2414 raft_consensus.cc:493] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:13.820595 2414 raft_consensus.cc:515] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:13.821354 2414 leader_election.cc:290] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.824831 772 raft_consensus.cc:2468] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:13.827226 767 raft_consensus.cc:2468] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:13.831718 2564 raft_consensus.cc:493] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.831912 2564 raft_consensus.cc:515] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.832993 2564 leader_election.cc:290] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:13.838653 1172 tablet_bootstrap.cc:492] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:13.838882 1172 ts_tablet_manager.cc:1403] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.151s user 0.003s sys 0.015s
I20260505 07:54:13.839407 2276 ts_tablet_manager.cc:1929] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.839520 2276 log.cc:1199] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/7001482d90364dbdb8b9c5bfb98e1cd4
I20260505 07:54:13.839921 2276 ts_tablet_manager.cc:1950] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:13.839897 1172 raft_consensus.cc:359] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.840338 1172 raft_consensus.cc:385] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.840449 1172 raft_consensus.cc:740] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.840729 1172 consensus_queue.cc:260] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.841275 1172 ts_tablet_manager.cc:1434] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:13.841605 1172 tablet_bootstrap.cc:492] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:13.842921 2447 ts_tablet_manager.cc:1929] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:13.843060 2447 log.cc:1199] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/7e707143a4e64725aa66db05b24999a6
I20260505 07:54:13.843545 2447 ts_tablet_manager.cc:1950] T 7e707143a4e64725aa66db05b24999a6 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:13.847337 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6607b7428044e428af9d765c9c9b3ba" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:13.848945 718 leader_election.cc:343] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 9a06f77255544d7dabb71a4f70ada809
I20260505 07:54:13.847146 854 leader_election.cc:304] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:13.849813 1176 tablet_bootstrap.cc:492] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.850171 1176 ts_tablet_manager.cc:1403] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.051s user 0.004s sys 0.000s
I20260505 07:54:13.850279 2564 raft_consensus.cc:2804] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:13.850419 2564 raft_consensus.cc:697] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
W20260505 07:54:13.850612 854 leader_election.cc:343] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.850733 2564 consensus_queue.cc:237] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.851023 2414 consensus_queue.cc:1048] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.851459 1176 raft_consensus.cc:359] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.851634 1176 raft_consensus.cc:385] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.851719 1176 raft_consensus.cc:740] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.851958 1176 consensus_queue.cc:260] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.852560 1176 ts_tablet_manager.cc:1434] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:13.852837 1176 tablet_bootstrap.cc:492] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.854984 1176 tablet_bootstrap.cc:654] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.859078 2749 consensus_queue.cc:1048] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.872566 1172 tablet_bootstrap.cc:654] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.883018 1176 tablet_bootstrap.cc:492] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.883186 1176 ts_tablet_manager.cc:1403] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.001s
I20260505 07:54:13.888362 2505 consensus_queue.cc:1048] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:13.786785 991 consensus_peers.cc:597] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0db162dfad2448e8ac428b3d3fbe17f7. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:13.897226 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 110466us
I20260505 07:54:13.899721 2844 raft_consensus.cc:493] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:13.899891 2844 raft_consensus.cc:515] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:13.899896 1176 raft_consensus.cc:359] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.900089 1176 raft_consensus.cc:385] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.900168 1176 raft_consensus.cc:740] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.789825 1100 heartbeater.cc:461] Registering TS with master...
I20260505 07:54:13.901971 1172 tablet_bootstrap.cc:492] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:13.902635 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a06f77255544d7dabb71a4f70ada809" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.902127 1172 ts_tablet_manager.cc:1403] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.002s
I20260505 07:54:13.902838 772 raft_consensus.cc:2468] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:13.903910 988 leader_election.cc:304] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.904218 2742 raft_consensus.cc:2804] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:13.904281 2742 raft_consensus.cc:493] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:13.904335 2742 raft_consensus.cc:3060] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:13.905642 2844 leader_election.cc:290] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.908077 1172 raft_consensus.cc:359] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.908259 1172 raft_consensus.cc:385] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.908362 1172 raft_consensus.cc:740] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.908382 2844 raft_consensus.cc:493] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:13.908604 1172 consensus_queue.cc:260] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.908576 2844 raft_consensus.cc:515] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:13.909147 1172 ts_tablet_manager.cc:1434] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.000s sys 0.000s
I20260505 07:54:13.909443 1172 tablet_bootstrap.cc:492] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:13.900408 1176 consensus_queue.cc:260] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:13.909987 988 leader_election.cc:343] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 12501fdf2cc5491cb1a06db490e70810
I20260505 07:54:13.910585 1176 ts_tablet_manager.cc:1434] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.027s user 0.002s sys 0.000s
I20260505 07:54:13.910931 1176 tablet_bootstrap.cc:492] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.911155 2844 leader_election.cc:290] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.912802 1176 tablet_bootstrap.cc:654] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.790366 2778 raft_consensus.cc:493] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.791383 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3462dcc6391741e59dcb5cd08527e50f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.833205 1878 raft_consensus.cc:515] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.919991 1878 leader_election.cc:290] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.833720 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6607b7428044e428af9d765c9c9b3ba" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.920284 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7d8265c05b74b28959d0de41fad1fde" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:13.920460 1047 raft_consensus.cc:2393] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:13.920498 767 raft_consensus.cc:3060] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.842500 2268 ts_tablet_manager.cc:1929] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:13.921060 2268 log.cc:1199] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/a7fe9c7199cc4039ae877dc3d4aab17e
I20260505 07:54:13.921401 2268 ts_tablet_manager.cc:1950] T a7fe9c7199cc4039ae877dc3d4aab17e P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:13.889492 2307 raft_consensus.cc:493] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:13.921559 2307 raft_consensus.cc:515] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:13.922122 2307 leader_election.cc:290] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.890318 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a06f77255544d7dabb71a4f70ada809" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.922488 1176 tablet_bootstrap.cc:492] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.922628 1176 ts_tablet_manager.cc:1403] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.000s
W20260505 07:54:13.923448 988 leader_election.cc:343] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 0459f2d45a2d44f1a73e0a0fff53866c
I20260505 07:54:13.923502 2778 raft_consensus.cc:515] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.923604 1176 raft_consensus.cc:359] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.923826 1176 raft_consensus.cc:385] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.923946 1176 raft_consensus.cc:740] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.928898 1054 raft_consensus.cc:2468] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 2.
I20260505 07:54:13.933473 918 raft_consensus.cc:2468] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:13.933918 718 leader_election.cc:304] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.934191 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 108) took 6152 ms. Trace:
I20260505 07:54:13.934332 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 162) took 4440 ms. Trace:
I20260505 07:54:13.934329 2276 rpcz_store.cc:276] 0505 07:54:07.781483 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.798908 (+ 17425us) service_pool.cc:224] Handling call
0505 07:54:13.636011 (+5837103us) ts_tablet_manager.cc:1151] Deleting tablet 7001482d90364dbdb8b9c5bfb98e1cd4
0505 07:54:13.636023 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.731406 (+ 95383us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.839306 (+107900us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.934155 (+ 94849us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5836975}
I20260505 07:54:13.934433 2447 rpcz_store.cc:276] 0505 07:54:09.493457 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.493492 (+ 35us) service_pool.cc:224] Handling call
0505 07:54:13.712580 (+4219088us) ts_tablet_manager.cc:1151] Deleting tablet 7e707143a4e64725aa66db05b24999a6
0505 07:54:13.712589 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.731569 (+ 18980us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.842814 (+111245us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.934283 (+ 91469us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4218890}
I20260505 07:54:13.934674 2276 tablet_replica.cc:333] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:13.934743 2447 tablet_replica.cc:333] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:13.934803 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 7001482d90364dbdb8b9c5bfb98e1cd4 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]) successfully deleted
I20260505 07:54:13.934890 2276 raft_consensus.cc:2243] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:13.935005 2447 raft_consensus.cc:2243] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:13.935070 2276 raft_consensus.cc:2272] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.935163 2447 raft_consensus.cc:2272] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:13.935585 2276 ts_tablet_manager.cc:1916] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.938369 2673 raft_consensus.cc:2749] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:13.938738 2414 raft_consensus.cc:2804] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:13.938913 2414 raft_consensus.cc:697] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:13.939260 2414 consensus_queue.cc:237] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.940706 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 7e707143a4e64725aa66db05b24999a6 (table default.Table_f0316c88590245aea8815b3b08e89f45 [id=c3a480b747de444aa45b771244908352]) successfully deleted
I20260505 07:54:13.941027 2447 ts_tablet_manager.cc:1916] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:13.941874 2505 raft_consensus.cc:493] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:13.942011 2505 raft_consensus.cc:515] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:13.942216 1172 tablet_bootstrap.cc:654] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.942683 2505 leader_election.cc:290] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:13.943389 854 leader_election.cc:343] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 0459f2d45a2d44f1a73e0a0fff53866c
I20260505 07:54:13.943802 1176 consensus_queue.cc:260] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.944353 1878 consensus_queue.cc:1048] T a1175f7deec14fe39098ec89ab3bc049 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:13.947083 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0459f2d45a2d44f1a73e0a0fff53866c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:13.947237 1054 raft_consensus.cc:2468] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:13.947628 856 leader_election.cc:304] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.947881 2752 raft_consensus.cc:2804] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:13.947955 2752 raft_consensus.cc:493] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:13.948038 2752 raft_consensus.cc:3060] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:13.948057 1176 ts_tablet_manager.cc:1434] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.025s user 0.002s sys 0.000s
I20260505 07:54:13.948339 1176 tablet_bootstrap.cc:492] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.949966 1176 tablet_bootstrap.cc:654] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.949963 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0459f2d45a2d44f1a73e0a0fff53866c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.959690 1050 raft_consensus.cc:2468] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:13.960408 856 leader_election.cc:304] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.960935 2749 raft_consensus.cc:2804] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.961014 2749 raft_consensus.cc:493] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.961086 2749 raft_consensus.cc:3060] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.963219 1176 tablet_bootstrap.cc:492] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.963405 1176 ts_tablet_manager.cc:1403] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260505 07:54:13.964193 1176 raft_consensus.cc:359] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.964401 1176 raft_consensus.cc:385] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.964509 1176 raft_consensus.cc:740] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.965451 1172 tablet_bootstrap.cc:492] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:13.965595 1172 ts_tablet_manager.cc:1403] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.056s user 0.004s sys 0.002s
I20260505 07:54:13.966372 1172 raft_consensus.cc:359] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.966529 1172 raft_consensus.cc:385] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.966588 1172 raft_consensus.cc:740] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.966815 1172 consensus_queue.cc:260] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.967371 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6607b7428044e428af9d765c9c9b3ba" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:13.967551 2778 leader_election.cc:290] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:13.967566 772 raft_consensus.cc:2468] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:13.967953 988 leader_election.cc:304] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:13.967999 1172 ts_tablet_manager.cc:1434] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:13.968324 2302 raft_consensus.cc:2804] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:13.968394 2302 raft_consensus.cc:493] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:13.968456 2302 raft_consensus.cc:3060] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:13.968475 1172 tablet_bootstrap.cc:492] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:13.970341 1172 tablet_bootstrap.cc:654] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:13.964859 1176 consensus_queue.cc:260] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.976465 1176 ts_tablet_manager.cc:1434] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.013s user 0.000s sys 0.002s
I20260505 07:54:13.976830 1176 tablet_bootstrap.cc:492] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:13.978065 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:13.978609 1176 tablet_bootstrap.cc:654] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:13.983071 991 leader_election.cc:343] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 9a06f77255544d7dabb71a4f70ada809
W20260505 07:54:13.983423 991 leader_election.cc:343] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 12501fdf2cc5491cb1a06db490e70810
I20260505 07:54:13.983549 991 leader_election.cc:304] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:13.984171 1878 raft_consensus.cc:2749] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:13.984666 991 leader_election.cc:343] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: a7d8265c05b74b28959d0de41fad1fde
I20260505 07:54:13.985561 1176 tablet_bootstrap.cc:492] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:13.985793 1176 ts_tablet_manager.cc:1403] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.002s
I20260505 07:54:13.986725 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6607b7428044e428af9d765c9c9b3ba" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:13.986846 1172 tablet_bootstrap.cc:492] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:13.987004 1172 ts_tablet_manager.cc:1403] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.002s
I20260505 07:54:13.987795 1172 raft_consensus.cc:359] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.987962 1172 raft_consensus.cc:385] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.988013 1172 raft_consensus.cc:740] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.988197 1172 consensus_queue.cc:260] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.988989 1176 raft_consensus.cc:359] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.989315 1176 raft_consensus.cc:385] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:13.989416 1176 raft_consensus.cc:740] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:13.989708 1176 consensus_queue.cc:260] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.990597 1176 ts_tablet_manager.cc:1434] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:54:13.990902 1176 tablet_bootstrap.cc:492] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:54:13.993095 717 leader_election.cc:343] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:13.994100 717 leader_election.cc:304] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:13.994477 2853 raft_consensus.cc:493] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:13.995100 2853 raft_consensus.cc:515] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:13.997218 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3075e87e078f4d2aa1d329d18902354b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:13.997725 854 leader_election.cc:343] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.000114 2853 leader_election.cc:290] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.000172 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3075e87e078f4d2aa1d329d18902354b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.000656 856 leader_election.cc:343] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.000747 856 leader_election.cc:304] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.001349 2564 raft_consensus.cc:2749] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.002554 1176 tablet_bootstrap.cc:654] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.007885 1172 ts_tablet_manager.cc:1434] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.021s user 0.004s sys 0.000s
I20260505 07:54:14.008140 1172 tablet_bootstrap.cc:492] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.009490 1172 tablet_bootstrap.cc:654] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.009707 2414 raft_consensus.cc:493] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.009820 2414 raft_consensus.cc:515] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.010325 2414 leader_election.cc:290] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.010612 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37765950e67d466683551f67a35e9bdb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.010771 921 raft_consensus.cc:2468] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:14.011216 718 leader_election.cc:304] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.011466 2414 raft_consensus.cc:2804] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.011538 2414 raft_consensus.cc:493] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.011595 2414 raft_consensus.cc:3060] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.011845 1181 tablet_bootstrap.cc:492] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:14.012243 1181 ts_tablet_manager.cc:1403] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.232s user 0.003s sys 0.000s
I20260505 07:54:14.012440 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37765950e67d466683551f67a35e9bdb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.013052 717 leader_election.cc:343] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.014605 1176 tablet_bootstrap.cc:492] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.014729 1176 ts_tablet_manager.cc:1403] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.002s
I20260505 07:54:14.015419 1176 raft_consensus.cc:359] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.015575 1176 raft_consensus.cc:385] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.015653 1176 raft_consensus.cc:740] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.015833 1176 consensus_queue.cc:260] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.016325 1176 ts_tablet_manager.cc:1434] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:14.016584 1176 tablet_bootstrap.cc:492] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:14.017962 1181 raft_consensus.cc:359] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.018170 1181 raft_consensus.cc:385] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.018255 1181 raft_consensus.cc:740] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.018525 1172 tablet_bootstrap.cc:492] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.018647 1172 ts_tablet_manager.cc:1403] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260505 07:54:14.018514 1181 consensus_queue.cc:260] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.019140 1181 ts_tablet_manager.cc:1434] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260505 07:54:14.019317 1172 raft_consensus.cc:359] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.019474 1172 raft_consensus.cc:385] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.019516 1172 raft_consensus.cc:740] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.019706 1181 tablet_bootstrap.cc:492] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:14.019709 1172 consensus_queue.cc:260] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.020156 1172 ts_tablet_manager.cc:1434] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260505 07:54:14.020442 1172 tablet_bootstrap.cc:492] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.021700 1172 tablet_bootstrap.cc:654] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.026460 2752 raft_consensus.cc:515] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.027002 2752 leader_election.cc:290] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:14.029170 2749 raft_consensus.cc:515] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.029502 907 raft_consensus.cc:2393] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:14.029675 2749 leader_election.cc:290] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.029928 918 raft_consensus.cc:2393] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 2: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:14.029946 1176 tablet_bootstrap.cc:654] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.030607 2276 ts_tablet_manager.cc:1929] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:14.030690 2276 log.cc:1199] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/02412333633a4cdebe95a144fa6c363c
I20260505 07:54:14.030956 2674 raft_consensus.cc:2749] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.031041 2276 ts_tablet_manager.cc:1950] T 02412333633a4cdebe95a144fa6c363c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:14.031116 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6607b7428044e428af9d765c9c9b3ba" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.031266 777 raft_consensus.cc:3060] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:14.031778 854 leader_election.cc:343] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 0459f2d45a2d44f1a73e0a0fff53866c
W20260505 07:54:14.035672 1317 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 68e8f9f7b30f4912af916c15f9ae25d8 consensus state has no leader
I20260505 07:54:14.038686 2673 raft_consensus.cc:493] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.038805 2673 raft_consensus.cc:515] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.039350 2673 leader_election.cc:290] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.039768 1172 tablet_bootstrap.cc:492] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.039964 1172 ts_tablet_manager.cc:1403] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260505 07:54:14.040091 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429ccdb56f0b456483cf1debb5de0beb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.040261 918 raft_consensus.cc:2468] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:14.040596 718 leader_election.cc:304] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.040808 2674 raft_consensus.cc:2804] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.040864 2674 raft_consensus.cc:493] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.040923 2674 raft_consensus.cc:3060] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.040983 1172 raft_consensus.cc:359] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.041100 1172 raft_consensus.cc:385] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.041149 1172 raft_consensus.cc:740] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.041311 1172 consensus_queue.cc:260] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.041749 1172 ts_tablet_manager.cc:1434] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:14.041989 1172 tablet_bootstrap.cc:492] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.043206 1172 tablet_bootstrap.cc:654] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.048024 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:14.049391 1176 tablet_bootstrap.cc:492] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.049525 1176 ts_tablet_manager.cc:1403] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.033s user 0.002s sys 0.003s
I20260505 07:54:14.021646 1181 tablet_bootstrap.cc:654] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:14.026587 991 consensus_peers.cc:597] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 202a237ca580482c98803683a77d96d2. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:14.034595 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0459f2d45a2d44f1a73e0a0fff53866c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.053045 1050 raft_consensus.cc:3060] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:14.034659 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6607b7428044e428af9d765c9c9b3ba" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.053426 991 leader_election.cc:304] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.042757 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429ccdb56f0b456483cf1debb5de0beb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.030795 1053 raft_consensus.cc:2468] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.054325 2405 raft_consensus.cc:2749] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:14.055256 991 consensus_peers.cc:597] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7001482d90364dbdb8b9c5bfb98e1cd4. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:14.057228 717 leader_election.cc:343] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.057574 2749 raft_consensus.cc:493] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:14.057722 2749 raft_consensus.cc:515] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.058219 2749 leader_election.cc:290] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:14.058931 854 leader_election.cc:343] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 297537a92684425d8508f7b3b4c9d379
W20260505 07:54:14.059295 856 leader_election.cc:343] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 297537a92684425d8508f7b3b4c9d379
I20260505 07:54:14.059378 856 leader_election.cc:304] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.059502 1176 raft_consensus.cc:359] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.059679 1176 raft_consensus.cc:385] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.059734 1176 raft_consensus.cc:740] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.059860 2752 raft_consensus.cc:2749] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:14.059935 1176 consensus_queue.cc:260] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.060432 1176 ts_tablet_manager.cc:1434] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20260505 07:54:14.060635 2673 raft_consensus.cc:493] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.060700 1176 tablet_bootstrap.cc:492] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:14.060738 2673 raft_consensus.cc:515] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.061228 2673 leader_election.cc:290] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.062157 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f842d8ac6b2142ffa69945c6087be34b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.062260 1176 tablet_bootstrap.cc:654] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:14.062562 717 leader_election.cc:343] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.063778 1172 tablet_bootstrap.cc:492] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.063889 1172 ts_tablet_manager.cc:1403] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.000s
I20260505 07:54:14.063911 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f842d8ac6b2142ffa69945c6087be34b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.064092 918 raft_consensus.cc:2468] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
W20260505 07:54:14.064484 718 consensus_peers.cc:597] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a79578757824bfc9df129e3475e7af3. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:14.064611 1172 raft_consensus.cc:359] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.064742 1172 raft_consensus.cc:385] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.064715 718 leader_election.cc:304] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.064785 1172 raft_consensus.cc:740] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.064954 1172 consensus_queue.cc:260] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.065114 2833 raft_consensus.cc:2804] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.065196 2833 raft_consensus.cc:493] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.065266 2833 raft_consensus.cc:3060] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.065274 1172 ts_tablet_manager.cc:1434] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:14.065492 1172 tablet_bootstrap.cc:492] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.065744 2749 consensus_queue.cc:1048] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.066896 1172 tablet_bootstrap.cc:654] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.075305 1176 tablet_bootstrap.cc:492] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.075438 1176 ts_tablet_manager.cc:1403] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260505 07:54:14.077332 1172 tablet_bootstrap.cc:492] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.077463 1172 ts_tablet_manager.cc:1403] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20260505 07:54:14.078151 1172 raft_consensus.cc:359] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.078307 1172 raft_consensus.cc:385] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.078373 1172 raft_consensus.cc:740] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.078577 1172 consensus_queue.cc:260] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.079044 1172 ts_tablet_manager.cc:1434] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:14.079293 1172 tablet_bootstrap.cc:492] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.079936 1176 raft_consensus.cc:359] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.080107 1176 raft_consensus.cc:385] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.080163 1176 raft_consensus.cc:740] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.080339 1176 consensus_queue.cc:260] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.082518 1176 ts_tablet_manager.cc:1434] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260505 07:54:14.082798 1176 tablet_bootstrap.cc:492] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:14.089522 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 109) took 6307 ms. Trace:
I20260505 07:54:14.089608 2276 rpcz_store.cc:276] 0505 07:54:07.781567 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.799042 (+ 17475us) service_pool.cc:224] Handling call
0505 07:54:13.934600 (+6135558us) ts_tablet_manager.cc:1151] Deleting tablet 02412333633a4cdebe95a144fa6c363c
0505 07:54:13.934611 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.026428 (+ 91817us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.030538 (+ 4110us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.089502 (+ 58964us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6135406}
I20260505 07:54:14.089972 2276 tablet_replica.cc:333] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:14.090138 2276 raft_consensus.cc:2243] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.090276 2276 raft_consensus.cc:2272] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.090297 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 02412333633a4cdebe95a144fa6c363c (table default.Table_0654ae2daf60435697eeca15ba0c19d7 [id=13d54930c52f4e9eae01bc80a3f1d565]) successfully deleted
I20260505 07:54:14.090732 2276 ts_tablet_manager.cc:1916] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.090967 2752 consensus_queue.cc:1048] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.095388 1172 tablet_bootstrap.cc:654] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.096258 1176 tablet_bootstrap.cc:654] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.108068 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:14.108695 1181 tablet_bootstrap.cc:492] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:14.108912 1181 ts_tablet_manager.cc:1403] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.089s user 0.002s sys 0.002s
I20260505 07:54:14.109247 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:14.109675 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:14.109962 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:14.110201 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:14.110461 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:14.110627 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:14.110865 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:14.111159 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:14.111550 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:14.111930 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:14.112151 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:14.112319 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:14.112516 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:14.112684 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:14.112507 1181 raft_consensus.cc:359] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.113327 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:14.113487 1181 raft_consensus.cc:385] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.113595 1181 raft_consensus.cc:740] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.113773 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:14.113849 1181 consensus_queue.cc:260] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.114182 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:14.114530 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:14.114934 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:14.115809 1181 ts_tablet_manager.cc:1434] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:54:14.116441 1181 tablet_bootstrap.cc:492] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:14.118083 1181 tablet_bootstrap.cc:654] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.129880 2276 ts_tablet_manager.cc:1929] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:14.130026 2276 log.cc:1199] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/36b33d265ba04e93993c7e0a4efea1a4
I20260505 07:54:14.130508 2276 ts_tablet_manager.cc:1950] T 36b33d265ba04e93993c7e0a4efea1a4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:14.131462 1181 tablet_bootstrap.cc:492] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:14.131675 1181 ts_tablet_manager.cc:1403] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.001s
I20260505 07:54:14.132433 1181 raft_consensus.cc:359] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.132706 1181 raft_consensus.cc:385] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.132808 1181 raft_consensus.cc:740] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.133052 1181 consensus_queue.cc:260] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.133245 2280 raft_consensus.cc:2749] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.133605 1181 ts_tablet_manager.cc:1434] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:14.133688 2447 ts_tablet_manager.cc:1929] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:14.133818 2447 log.cc:1199] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/0db162dfad2448e8ac428b3d3fbe17f7
I20260505 07:54:14.133906 1181 tablet_bootstrap.cc:492] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:14.134157 2447 ts_tablet_manager.cc:1950] T 0db162dfad2448e8ac428b3d3fbe17f7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:14.135787 1181 tablet_bootstrap.cc:654] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.138626 2139 ts_manager.cc:194] Registered new tserver with Master: d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.140965 2136 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" instance_seqno: 1777967632042903) as {username='slave'} at 127.0.105.2:51589; Asking this server to re-register.
I20260505 07:54:14.145066 967 heartbeater.cc:461] Registering TS with master...
I20260505 07:54:14.133061 1048 raft_consensus.cc:2468] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.147368 1181 tablet_bootstrap.cc:492] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:14.147534 1181 ts_tablet_manager.cc:1403] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.003s
I20260505 07:54:14.148538 1181 raft_consensus.cc:359] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.148823 1181 raft_consensus.cc:385] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.148931 1181 raft_consensus.cc:740] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.149169 1181 consensus_queue.cc:260] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.149648 1181 ts_tablet_manager.cc:1434] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:14.152312 2136 ts_manager.cc:194] Registered new tserver with Master: d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.152271 2799 consensus_queue.cc:1048] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.155334 1181 tablet_bootstrap.cc:492] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:14.157649 2752 raft_consensus.cc:493] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.157766 2752 raft_consensus.cc:515] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.158317 2752 leader_election.cc:290] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.158757 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b528d6c36ac4920b6ea0a6e26cca269" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:14.159349 854 leader_election.cc:343] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.159719 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b528d6c36ac4920b6ea0a6e26cca269" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.160156 856 leader_election.cc:343] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.160284 856 leader_election.cc:304] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.160624 2752 raft_consensus.cc:2749] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.162117 1181 tablet_bootstrap.cc:654] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.165745 2752 consensus_queue.cc:1048] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.167198 1176 tablet_bootstrap.cc:492] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.167335 1176 ts_tablet_manager.cc:1403] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.085s user 0.005s sys 0.008s
I20260505 07:54:14.168098 1176 raft_consensus.cc:359] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.168260 1176 raft_consensus.cc:385] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.168318 1176 raft_consensus.cc:740] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.169286 2280 raft_consensus.cc:493] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.169402 2280 raft_consensus.cc:515] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.169914 2280 leader_election.cc:290] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.170238 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9b7c6e9463342a5a73af12da4447f89" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.170423 918 raft_consensus.cc:2468] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:14.170928 718 leader_election.cc:304] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.171183 2280 raft_consensus.cc:2804] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.171260 2280 raft_consensus.cc:493] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.171375 2280 raft_consensus.cc:3060] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.174188 1172 tablet_bootstrap.cc:492] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.174312 1172 ts_tablet_manager.cc:1403] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.095s user 0.001s sys 0.012s
I20260505 07:54:14.171721 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9b7c6e9463342a5a73af12da4447f89" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.176687 717 leader_election.cc:343] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.177423 1172 raft_consensus.cc:359] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.177590 1172 raft_consensus.cc:385] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.177656 1172 raft_consensus.cc:740] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.177695 2505 consensus_queue.cc:1048] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.179111 2505 raft_consensus.cc:493] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.179230 2505 raft_consensus.cc:515] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.179741 2505 leader_election.cc:290] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.179977 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abaa596871bb4a82aefc317932e91d35" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.180246 1172 consensus_queue.cc:260] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:14.180501 854 leader_election.cc:343] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.181258 1172 ts_tablet_manager.cc:1434] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20260505 07:54:14.181519 1172 tablet_bootstrap.cc:492] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.183068 1172 tablet_bootstrap.cc:654] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.183911 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abaa596871bb4a82aefc317932e91d35" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.184026 1176 consensus_queue.cc:260] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.184545 1176 ts_tablet_manager.cc:1434] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.017s user 0.002s sys 0.000s
W20260505 07:54:14.184661 856 leader_election.cc:343] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.184767 856 leader_election.cc:304] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.184818 1176 tablet_bootstrap.cc:492] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:14.185153 2505 raft_consensus.cc:2749] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.188035 2505 raft_consensus.cc:493] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.188138 2505 raft_consensus.cc:515] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.188588 2505 leader_election.cc:290] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.189260 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60772a9697ba417b879dc6025eeb8eec" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.189718 856 leader_election.cc:343] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.192551 2505 raft_consensus.cc:493] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.192651 2505 raft_consensus.cc:515] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.193125 2505 leader_election.cc:290] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.193758 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:14.193795 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbfb2345a1a4b218c2dddbae6667e47" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.194226 856 leader_election.cc:343] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.194926 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:14.199389 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 110) took 6417 ms. Trace:
I20260505 07:54:14.199483 2276 rpcz_store.cc:276] 0505 07:54:07.781636 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.799203 (+ 17567us) service_pool.cc:224] Handling call
0505 07:54:14.089912 (+6290709us) ts_tablet_manager.cc:1151] Deleting tablet 36b33d265ba04e93993c7e0a4efea1a4
0505 07:54:14.089932 (+ 20us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.126884 (+ 36952us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.129830 (+ 2946us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.199371 (+ 69541us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6290603}
I20260505 07:54:14.199810 2276 tablet_replica.cc:333] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:14.199994 2276 raft_consensus.cc:2243] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.200138 2276 raft_consensus.cc:2272] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.200639 2276 ts_tablet_manager.cc:1916] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.201361 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60772a9697ba417b879dc6025eeb8eec" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.200026 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 36b33d265ba04e93993c7e0a4efea1a4 (table default.Table_0b3a73b640a644579fa864cce1a7ecbe [id=6c1e059a94014956ab0b1d6c750d8494]) successfully deleted
I20260505 07:54:14.202397 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:14.202574 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:14.202903 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:14.203049 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:14.203269 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbfb2345a1a4b218c2dddbae6667e47" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.203562 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
W20260505 07:54:14.203651 854 leader_election.cc:343] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.203735 854 leader_election.cc:304] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.203919 2505 raft_consensus.cc:2749] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:14.204113 854 leader_election.cc:343] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.204221 854 leader_election.cc:304] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.204411 2505 raft_consensus.cc:2749] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.205013 1176 tablet_bootstrap.cc:654] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.209185 2650 consensus_queue.cc:1048] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.217373 1181 tablet_bootstrap.cc:492] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:14.217520 1181 ts_tablet_manager.cc:1403] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.062s user 0.004s sys 0.000s
I20260505 07:54:14.218895 1172 tablet_bootstrap.cc:492] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.219383 1181 raft_consensus.cc:359] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.219700 1172 ts_tablet_manager.cc:1403] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.038s user 0.000s sys 0.004s
I20260505 07:54:14.219573 1181 raft_consensus.cc:385] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.220074 1181 raft_consensus.cc:740] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.220777 1172 raft_consensus.cc:359] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.220960 1172 raft_consensus.cc:385] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.221020 1172 raft_consensus.cc:740] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.221256 1172 consensus_queue.cc:260] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.221716 1172 ts_tablet_manager.cc:1434] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:14.221784 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:14.221978 1172 tablet_bootstrap.cc:492] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.222429 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:14.223534 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:14.224073 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:14.226467 1172 tablet_bootstrap.cc:654] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.227478 2742 raft_consensus.cc:515] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.228117 2742 leader_election.cc:290] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.228873 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 144) took 4421 ms. Trace:
I20260505 07:54:14.228955 2268 rpcz_store.cc:276] 0505 07:54:09.807614 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.810082 (+ 2468us) service_pool.cc:224] Handling call
0505 07:54:13.701456 (+3891374us) ts_tablet_manager.cc:1151] Deleting tablet a7fe9c7199cc4039ae877dc3d4aab17e
0505 07:54:13.701471 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:13.742002 (+ 40531us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:13.842443 (+100441us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.228858 (+386415us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3891208}
I20260505 07:54:14.229202 2268 tablet_replica.cc:333] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:14.229367 2268 raft_consensus.cc:2243] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:14.229573 2872 raft_consensus.cc:493] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.229676 2268 raft_consensus.cc:2272] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.229683 2872 raft_consensus.cc:515] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.230144 2268 ts_tablet_manager.cc:1916] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.230212 2872 leader_election.cc:290] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.230702 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3e076892234c1494b1a5c13cbfd2e2" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.230854 921 raft_consensus.cc:2468] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:14.231186 718 leader_election.cc:304] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.231395 2872 raft_consensus.cc:2804] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.231448 2872 raft_consensus.cc:493] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.231498 2872 raft_consensus.cc:3060] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.231689 2670 raft_consensus.cc:515] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.232298 2670 leader_election.cc:290] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.233875 2505 raft_consensus.cc:493] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.233973 2505 raft_consensus.cc:515] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.234421 2505 leader_election.cc:290] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.234733 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bf49b787ef4de095448751464708a4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.234938 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet a7fe9c7199cc4039ae877dc3d4aab17e (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]) successfully deleted
I20260505 07:54:14.235320 1052 raft_consensus.cc:2393] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 2: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
W20260505 07:54:14.235328 854 leader_election.cc:343] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:14.235664 991 leader_election.cc:343] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 9a06f77255544d7dabb71a4f70ada809
I20260505 07:54:14.235885 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4dd3ece5cb34ceb82feeb67e75da0b5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.236024 921 raft_consensus.cc:3060] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.238637 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a06f77255544d7dabb71a4f70ada809" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.238768 772 raft_consensus.cc:3060] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:14.239398 717 leader_election.cc:304] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.240258 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4dd3ece5cb34ceb82feeb67e75da0b5" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.240410 1052 raft_consensus.cc:3060] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.240924 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bf49b787ef4de095448751464708a4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.241761 856 leader_election.cc:343] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.241863 856 leader_election.cc:304] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.241883 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3e076892234c1494b1a5c13cbfd2e2" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.242184 2505 raft_consensus.cc:2749] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:14.242292 717 leader_election.cc:343] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.243512 2888 tablet.cc:1724] T 0e4e5af08ddc4c77b5640b8295b236b1 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:14.245201 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:14.245792 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:14.246021 2673 raft_consensus.cc:493] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.246133 2673 raft_consensus.cc:515] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.246625 2673 leader_election.cc:290] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:14.247380 718 leader_election.cc:343] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: a7fe9c7199cc4039ae877dc3d4aab17e
I20260505 07:54:14.248008 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
W20260505 07:54:14.248518 717 leader_election.cc:343] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: a7fe9c7199cc4039ae877dc3d4aab17e
I20260505 07:54:14.248651 717 leader_election.cc:304] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
W20260505 07:54:14.248816 991 consensus_peers.cc:597] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8a60c015136246a3b02f70254c0bb67f. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:14.248894 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:14.249137 2673 raft_consensus.cc:2749] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.253846 2778 raft_consensus.cc:493] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.253985 2778 raft_consensus.cc:515] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.254627 2778 leader_election.cc:290] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.255330 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423c22891bed462d89e9ec51a079f0f5" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.255518 771 raft_consensus.cc:2468] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:14.256222 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423c22891bed462d89e9ec51a079f0f5" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.256345 988 leader_election.cc:304] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.256386 907 raft_consensus.cc:2468] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:14.256686 2778 raft_consensus.cc:2804] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.256773 2778 raft_consensus.cc:493] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.256846 2778 raft_consensus.cc:3060] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.257449 2887 tablet.cc:1724] T 0e4e5af08ddc4c77b5640b8295b236b1 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:14.263710 1181 consensus_queue.cc:260] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.264652 1181 ts_tablet_manager.cc:1434] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.047s user 0.003s sys 0.000s
I20260505 07:54:14.264999 1181 tablet_bootstrap.cc:492] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:14.266656 1181 tablet_bootstrap.cc:654] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.269039 1176 tablet_bootstrap.cc:492] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.269176 1176 ts_tablet_manager.cc:1403] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.084s user 0.002s sys 0.001s
I20260505 07:54:14.269383 2505 consensus_queue.cc:1048] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.269942 1176 raft_consensus.cc:359] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.270126 1176 raft_consensus.cc:385] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.270179 1176 raft_consensus.cc:740] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.270447 1176 consensus_queue.cc:260] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.270920 1176 ts_tablet_manager.cc:1434] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:14.271193 1176 tablet_bootstrap.cc:492] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:14.273339 1176 tablet_bootstrap.cc:654] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.281797 2853 consensus_queue.cc:1048] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.281957 1181 tablet_bootstrap.cc:492] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:14.282073 1181 ts_tablet_manager.cc:1403] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.001s
I20260505 07:54:14.282667 1181 raft_consensus.cc:359] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.284449 1176 tablet_bootstrap.cc:492] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.284608 1176 ts_tablet_manager.cc:1403] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260505 07:54:14.285264 1176 raft_consensus.cc:359] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.285408 1176 raft_consensus.cc:385] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.285465 1176 raft_consensus.cc:740] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.285671 1176 consensus_queue.cc:260] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.286073 1176 ts_tablet_manager.cc:1434] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:14.293412 1176 tablet_bootstrap.cc:492] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:14.294961 1176 tablet_bootstrap.cc:654] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.295854 2650 raft_consensus.cc:493] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:14.298637 2799 consensus_queue.cc:1048] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.300017 2650 raft_consensus.cc:515] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:14.301132 988 leader_election.cc:343] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 1ef4f516e64a44fd8cd33674fb808067
I20260505 07:54:14.301450 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:14.301661 2650 leader_election.cc:290] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.301810 1181 raft_consensus.cc:385] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.302327 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:14.302773 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:14.302976 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:14.303180 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:14.303707 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
W20260505 07:54:14.302094 991 leader_election.cc:343] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 1ef4f516e64a44fd8cd33674fb808067
I20260505 07:54:14.304035 991 leader_election.cc:304] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.304415 2799 raft_consensus.cc:2749] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:14.301872 1181 raft_consensus.cc:740] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.306747 1181 consensus_queue.cc:260] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.309274 1181 ts_tablet_manager.cc:1434] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.027s user 0.001s sys 0.001s
I20260505 07:54:14.316752 1352 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:14.323650 1181 tablet_bootstrap.cc:492] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:14.325729 2302 raft_consensus.cc:515] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.326468 2302 leader_election.cc:290] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.329862 2276 ts_tablet_manager.cc:1929] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:14.329994 2276 log.cc:1199] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/46274a44a06b4306b5dac43c66c85a80
I20260505 07:54:14.330336 2276 ts_tablet_manager.cc:1950] T 46274a44a06b4306b5dac43c66c85a80 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
W20260505 07:54:14.330647 2672 consensus_meta.cc:287] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent flushing consensus metadata: real 0.529s user 0.002s sys 0.001s
I20260505 07:54:14.330763 2672 raft_consensus.cc:515] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.331313 2672 leader_election.cc:290] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.331283 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6607b7428044e428af9d765c9c9b3ba" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.331545 907 raft_consensus.cc:2393] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:14.331808 1054 raft_consensus.cc:2393] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d2799f5c6b4f4336b976ca9bbb458533 in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:14.333236 2892 tablet.cc:1724] T 0e4e5af08ddc4c77b5640b8295b236b1 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:14.336982 2853 raft_consensus.cc:493] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.337116 2853 raft_consensus.cc:515] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.337694 2853 leader_election.cc:290] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.336836 1176 tablet_bootstrap.cc:492] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.338169 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e06d8815034d91986cef12964860ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.338316 781 raft_consensus.cc:2468] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:14.338390 1176 ts_tablet_manager.cc:1403] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.045s user 0.004s sys 0.004s
I20260505 07:54:14.338651 854 leader_election.cc:304] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.338863 2853 raft_consensus.cc:2804] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.338936 2853 raft_consensus.cc:493] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.339010 2853 raft_consensus.cc:3060] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.339953 2139 ts_manager.cc:194] Registered new tserver with Master: eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:14.340015 1176 raft_consensus.cc:359] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.340149 1176 raft_consensus.cc:385] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.340190 1176 raft_consensus.cc:740] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.340328 1176 consensus_queue.cc:260] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.344434 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3462dcc6391741e59dcb5cd08527e50f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:14.344954 717 leader_election.cc:343] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.346004 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e06d8815034d91986cef12964860ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.347308 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3462dcc6391741e59dcb5cd08527e50f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.347973 907 raft_consensus.cc:3060] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:14.348727 856 leader_election.cc:343] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.349319 2900 raft_consensus.cc:493] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:14.349604 921 raft_consensus.cc:2468] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.349653 2900 raft_consensus.cc:515] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.350165 718 leader_election.cc:304] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.350291 2900 leader_election.cc:290] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.350755 2900 raft_consensus.cc:2804] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.350701 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac5064f68d24ebe95b5a0b8e10f463e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.352843 2900 raft_consensus.cc:697] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:14.353137 2900 consensus_queue.cc:237] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:14.351016 718 leader_election.cc:343] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: cac5064f68d24ebe95b5a0b8e10f463e
I20260505 07:54:14.354981 717 leader_election.cc:304] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.356437 2799 consensus_queue.cc:1048] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.359519 2899 raft_consensus.cc:2749] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:14.365651 2856 raft_consensus.cc:493] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.365785 2856 raft_consensus.cc:515] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.366336 2856 leader_election.cc:290] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.366770 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a890d5e3f1af49b5aaa28c6c51cb3874" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.366899 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a890d5e3f1af49b5aaa28c6c51cb3874" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:14.367236 856 leader_election.cc:343] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:14.367375 854 leader_election.cc:343] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.367468 854 leader_election.cc:304] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.367884 2856 raft_consensus.cc:2749] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.368846 1172 tablet_bootstrap.cc:492] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.369010 1172 ts_tablet_manager.cc:1403] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.147s user 0.006s sys 0.006s
I20260505 07:54:14.369938 1172 raft_consensus.cc:359] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.370147 1172 raft_consensus.cc:385] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.370213 1172 raft_consensus.cc:740] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.370431 1172 consensus_queue.cc:260] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.372880 1176 ts_tablet_manager.cc:1434] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.034s user 0.000s sys 0.003s
I20260505 07:54:14.373221 1176 tablet_bootstrap.cc:492] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:14.376291 1172 ts_tablet_manager.cc:1434] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:54:14.376591 1172 tablet_bootstrap.cc:492] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.379660 1172 tablet_bootstrap.cc:654] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.390161 1176 tablet_bootstrap.cc:654] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.395128 1181 tablet_bootstrap.cc:654] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.395790 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6607b7428044e428af9d765c9c9b3ba" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.399173 988 leader_election.cc:304] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.399613 2650 raft_consensus.cc:2749] T b6607b7428044e428af9d765c9c9b3ba P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260505 07:54:14.399799 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 44469.2us
I20260505 07:54:14.408473 1181 tablet_bootstrap.cc:492] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:14.411469 1181 ts_tablet_manager.cc:1403] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.088s user 0.004s sys 0.000s
I20260505 07:54:14.412379 1172 tablet_bootstrap.cc:492] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.412554 1172 ts_tablet_manager.cc:1403] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.036s user 0.001s sys 0.005s
I20260505 07:54:14.413467 1172 raft_consensus.cc:359] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.413734 1172 raft_consensus.cc:385] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.413825 1172 raft_consensus.cc:740] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.414079 1172 consensus_queue.cc:260] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.414575 2856 raft_consensus.cc:493] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.414687 2856 raft_consensus.cc:515] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.415211 2856 leader_election.cc:290] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.415913 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a82ed2b87c487aac84194f2dcb0d6b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.415761 1172 ts_tablet_manager.cc:1434] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:14.416296 1172 tablet_bootstrap.cc:492] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:14.416467 854 leader_election.cc:343] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.418809 1172 tablet_bootstrap.cc:654] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.422322 2853 raft_consensus.cc:515] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.422919 2853 leader_election.cc:290] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.423207 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e06d8815034d91986cef12964860ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.423451 781 raft_consensus.cc:3060] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.425323 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:14.425748 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 111) took 6640 ms. Trace:
I20260505 07:54:14.425779 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:14.425863 2276 rpcz_store.cc:276] 0505 07:54:07.781699 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.799322 (+ 17623us) service_pool.cc:224] Handling call
0505 07:54:14.199769 (+6400447us) ts_tablet_manager.cc:1151] Deleting tablet 46274a44a06b4306b5dac43c66c85a80
0505 07:54:14.199778 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.232521 (+ 32743us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.329788 (+ 97267us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.421775 (+ 91987us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6400320}
I20260505 07:54:14.426213 2276 tablet_replica.cc:333] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:14.426424 2276 raft_consensus.cc:2243] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.426569 2276 raft_consensus.cc:2272] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.426630 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 46274a44a06b4306b5dac43c66c85a80 (table default.Table_614ea89c84d342f0a1c4912e8e77fe47 [id=f1230f25280e45269ffbd95bbe8dbbfc]) successfully deleted
I20260505 07:54:14.427081 2276 ts_tablet_manager.cc:1916] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.426591 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:14.427532 1176 tablet_bootstrap.cc:492] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.427572 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:14.430907 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:14.431229 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:14.431465 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:14.431715 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:14.431972 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:14.432276 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:14.432561 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:14.432844 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
W20260505 07:54:14.427561 767 consensus_meta.cc:287] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c: Time spent flushing consensus metadata: real 0.507s user 0.003s sys 0.000s
I20260505 07:54:14.433197 767 raft_consensus.cc:2468] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:14.433892 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:14.434034 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:14.434175 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:14.434612 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:14.434758 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:14.434903 2831 raft_consensus.cc:2749] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.435061 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:14.435454 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:14.435719 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:14.436331 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:14.436494 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:14.436691 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:14.436973 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:14.437232 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:14.437456 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:14.437783 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:14.438351 2831 raft_consensus.cc:493] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:14.438462 2831 raft_consensus.cc:515] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.439143 2831 leader_election.cc:290] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.440220 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e29ba55a846548679cb3b86305d0d6ed" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
W20260505 07:54:14.440646 718 leader_election.cc:343] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:54:14.427695 1176 ts_tablet_manager.cc:1403] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.055s user 0.000s sys 0.004s
W20260505 07:54:14.441298 717 leader_election.cc:343] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: e29ba55a846548679cb3b86305d0d6ed
I20260505 07:54:14.441449 717 leader_election.cc:304] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.441495 1176 raft_consensus.cc:359] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.441643 1176 raft_consensus.cc:385] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.441700 1176 raft_consensus.cc:740] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.441735 2901 raft_consensus.cc:2749] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:14.441857 1176 consensus_queue.cc:260] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.442301 1176 ts_tablet_manager.cc:1434] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:14.442550 1176 tablet_bootstrap.cc:492] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:14.444218 1176 tablet_bootstrap.cc:654] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.415908 1054 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a82ed2b87c487aac84194f2dcb0d6b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.422106 2268 ts_tablet_manager.cc:1929] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:14.426594 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e06d8815034d91986cef12964860ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.448225 2268 log.cc:1199] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/a0f2f46d0e7840458cb992ce2a2dbea2
I20260505 07:54:14.430696 1050 raft_consensus.cc:2468] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 2.
I20260505 07:54:14.433667 988 leader_election.cc:304] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:14.448487 2405 raft_consensus.cc:2749] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:14.448614 2268 ts_tablet_manager.cc:1950] T a0f2f46d0e7840458cb992ce2a2dbea2 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:14.448782 2405 raft_consensus.cc:2804] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.448897 2405 raft_consensus.cc:697] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:14.449115 2405 consensus_queue.cc:237] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.449190 1181 raft_consensus.cc:359] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.449295 1181 raft_consensus.cc:385] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.449337 1181 raft_consensus.cc:740] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
W20260505 07:54:14.449738 856 leader_election.cc:343] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.449890 856 leader_election.cc:304] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.450124 2856 raft_consensus.cc:2749] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:14.450278 856 leader_election.cc:343] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.450476 856 leader_election.cc:304] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.450706 2856 raft_consensus.cc:2804] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:14.450855 2856 raft_consensus.cc:697] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:14.451191 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:14.451192 2856 consensus_queue.cc:237] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.451781 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:14.452741 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:14.453006 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:14.453377 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:14.454028 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:14.454237 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:14.454478 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:14.454619 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:14.454907 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:14.455544 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:14.456068 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:14.456233 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:14.456591 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_0c1bd9ecad69401ab663ea12961bd890 [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 1 got 0
I20260505 07:54:14.456804 1353 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:14.458369 1172 tablet_bootstrap.cc:492] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.458498 1172 ts_tablet_manager.cc:1403] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.042s user 0.000s sys 0.004s
I20260505 07:54:14.458953 2853 raft_consensus.cc:493] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.459077 2853 raft_consensus.cc:515] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.459656 2853 leader_election.cc:290] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.459885 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd2e2514c72419390c1bb9bd13f1fb3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.460181 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd2e2514c72419390c1bb9bd13f1fb3" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.460407 854 leader_election.cc:343] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.460456 1172 raft_consensus.cc:359] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.460624 1172 raft_consensus.cc:385] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.460701 1172 raft_consensus.cc:740] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:54:14.460733 856 leader_election.cc:343] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.460816 856 leader_election.cc:304] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.460886 1172 consensus_queue.cc:260] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.461099 2752 raft_consensus.cc:2749] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.461357 1172 ts_tablet_manager.cc:1434] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:14.461622 1172 tablet_bootstrap.cc:492] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.462721 1181 consensus_queue.cc:260] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.462963 1358 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_8ecdd63342f547ab8e760480eca79fb1 [id=048ca74dfef1481d945932e17b785ce3] alter complete (version 2)
I20260505 07:54:14.467298 1172 tablet_bootstrap.cc:654] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.464576 1050 raft_consensus.cc:1275] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Refusing update from remote peer eac7e1e8caba43cdb4df80cbb80e891c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260505 07:54:14.468202 887 tablet_service.cc:1558] Processing DeleteTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.468384 2902 consensus_queue.cc:1048] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260505 07:54:14.470055 1181 ts_tablet_manager.cc:1434] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20260505 07:54:14.470052 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.470096 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 6da10815e0dc41a1a07b6786882fffd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.470052 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 with delete_type TABLET_DATA_DELETED (replaced by tablet d5083f98e8e14ccf99a19cdf2bc59018) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:14.469522 1358 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet b0d966bcd7f1419cbd090afc185ae78a (table default.Table_58eed87aa11e4e3fa7a7d18a135128a0 [id=3ae3bc457dcb4db1b5f4360cb9e5c81c], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet b0d966bcd7f1419cbd090afc185ae78a consensus state has no leader
I20260505 07:54:14.470861 899 tablet_service.cc:1558] Processing DeleteTablet for tablet d896e290167b4d0081e8e8b831c8ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.471464 899 tablet_service.cc:1558] Processing DeleteTablet for tablet abaa596871bb4a82aefc317932e91d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.471508 1181 tablet_bootstrap.cc:492] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:14.471530 918 raft_consensus.cc:1275] T 4e99f96a753047348371bee5c022684e P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Refusing update from remote peer eac7e1e8caba43cdb4df80cbb80e891c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260505 07:54:14.472002 1176 tablet_bootstrap.cc:492] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.472056 899 tablet_service.cc:1558] Processing DeleteTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.472131 1176 ts_tablet_manager.cc:1403] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.001s
I20260505 07:54:14.473217 1181 tablet_bootstrap.cc:654] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.473210 2831 consensus_queue.cc:1048] T 4e99f96a753047348371bee5c022684e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.473474 1176 raft_consensus.cc:359] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.473632 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 12501fdf2cc5491cb1a06db490e70810 with delete_type TABLET_DATA_DELETED (replaced by tablet d622612bdc404727a27b83769bf39738) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.473641 1176 raft_consensus.cc:385] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.473704 1176 raft_consensus.cc:740] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.475005 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 46274a44a06b4306b5dac43c66c85a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.475319 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 78a6470720984761a8606abeafbef992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.475317 894 tablet_service.cc:1558] Processing DeleteTablet for tablet f1058fcb9fa6457a85489049ca1d9ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.480053 1176 consensus_queue.cc:260] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.480510 1176 ts_tablet_manager.cc:1434] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260505 07:54:14.480773 1176 tablet_bootstrap.cc:492] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:14.483603 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.483865 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 003cd4472b424aacbba8c5f89900e228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.489362 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 141665fcd59048bf926acd0287f1853e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.489413 894 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a06f77255544d7dabb71a4f70ada809 with delete_type TABLET_DATA_DELETED (replaced by tablet 12841efcef6041d89fc6d7d9b90448c0) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.489632 887 tablet_service.cc:1558] Processing DeleteTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.489878 899 tablet_service.cc:1558] Processing DeleteTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e with delete_type TABLET_DATA_DELETED (replaced by tablet dd3fc8a9d9f04636b1f31bb88411c1da) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.490128 1354 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_b85a2b107c88498e84217252ca6cb947 [id=2c91868ddba84103b9a5c78201842a9e] alter complete (version 2)
I20260505 07:54:14.498898 1181 tablet_bootstrap.cc:492] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:14.499017 1181 ts_tablet_manager.cc:1403] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.028s user 0.004s sys 0.000s
I20260505 07:54:14.499780 1181 raft_consensus.cc:359] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.499934 1181 raft_consensus.cc:385] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.499975 1181 raft_consensus.cc:740] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.500142 1181 consensus_queue.cc:260] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.504963 2902 consensus_queue.cc:1048] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.507790 1181 ts_tablet_manager.cc:1434] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260505 07:54:14.508100 1181 tablet_bootstrap.cc:492] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:14.509598 1181 tablet_bootstrap.cc:654] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.511904 1176 tablet_bootstrap.cc:654] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:14.521023 2414 consensus_meta.cc:287] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c: Time spent flushing consensus metadata: real 0.509s user 0.002s sys 0.000s
I20260505 07:54:14.521129 2414 raft_consensus.cc:515] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.521660 2414 leader_election.cc:290] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.521693 2778 raft_consensus.cc:515] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.521965 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37765950e67d466683551f67a35e9bdb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.522107 921 raft_consensus.cc:3060] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.522255 2778 leader_election.cc:290] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.522464 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37765950e67d466683551f67a35e9bdb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.522600 1050 raft_consensus.cc:3060] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.523787 907 raft_consensus.cc:2468] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.525108 1050 raft_consensus.cc:2468] T 37765950e67d466683551f67a35e9bdb P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.525369 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423c22891bed462d89e9ec51a079f0f5" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.525501 912 raft_consensus.cc:3060] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.526024 921 raft_consensus.cc:2468] T 37765950e67d466683551f67a35e9bdb P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.526490 777 raft_consensus.cc:2468] T b6607b7428044e428af9d765c9c9b3ba P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
W20260505 07:54:14.527194 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38eee32a6fd84f89a1e88074f16d4242 consensus state has no leader
I20260505 07:54:14.527386 1349 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 180) took 2265 ms. Trace:
I20260505 07:54:14.527822 1052 raft_consensus.cc:2468] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.527913 912 raft_consensus.cc:2468] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:14.527467 1349 rpcz_store.cc:276] 0505 07:54:12.261543 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.659371 (+1397828us) service_pool.cc:224] Handling call
0505 07:54:13.659768 (+ 397us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:13.660010 (+ 242us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:13.660202 (+ 192us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:13.660278 (+ 76us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:14.047738 (+387460us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:14.527326 (+479588us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:13.660797 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.660904 (+ 107us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.660916 (+ 12us) write_op.cc:435] Acquired schema lock
0505 07:54:13.660919 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:13.661019 (+ 100us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.661030 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.661035 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:13.661095 (+ 60us) tablet.cc:701] Row locks acquired
0505 07:54:13.661098 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:13.661126 (+ 28us) write_op.cc:270] Start()
0505 07:54:13.661157 (+ 31us) write_op.cc:276] Timestamp: P: 1777967653661085 usec, L: 0
0505 07:54:13.661161 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:13.661319 (+ 158us) log.cc:844] Serialized 405 byte log entry
0505 07:54:14.046037 (+384718us) op_driver.cc:464] REPLICATION: finished
0505 07:54:14.046165 (+ 128us) write_op.cc:301] APPLY: starting
0505 07:54:14.046188 (+ 23us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:14.046233 (+ 45us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:14.047031 (+ 798us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:14.047093 (+ 62us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:14.047135 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:14.047146 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:14.047231 (+ 85us) log.cc:844] Serialized 23 byte log entry
0505 07:54:14.047299 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.047318 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:14.047326 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:14.047411 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.047416 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":43,"apply.run_cpu_time_us":508,"apply.run_wall_time_us":1243,"num_ops":1,"prepare.queue_time_us":50,"prepare.run_cpu_time_us":601,"prepare.run_wall_time_us":601,"replication_time_us":384835}]]}
I20260505 07:54:14.528370 991 leader_election.cc:304] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:14.528863 2902 raft_consensus.cc:493] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:14.528970 2902 raft_consensus.cc:515] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.529564 2902 leader_election.cc:290] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.530285 2674 raft_consensus.cc:515] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.530467 717 leader_election.cc:304] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.530782 2674 leader_election.cc:290] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.530908 2674 raft_consensus.cc:2804] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.531008 2778 raft_consensus.cc:2804] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.531008 2674 raft_consensus.cc:697] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:14.531100 2778 raft_consensus.cc:697] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:14.531215 2674 consensus_queue.cc:237] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.531277 2778 consensus_queue.cc:237] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.531337 1357 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 9f541eff399641dd81a0771186805493
I20260505 07:54:14.532011 1357 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 179) took 2271 ms. Trace:
I20260505 07:54:14.532054 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429ccdb56f0b456483cf1debb5de0beb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.532605 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 9f541eff399641dd81a0771186805493 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:13 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.532127 1357 rpcz_store.cc:276] 0505 07:54:12.260295 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.486240 (+1225945us) service_pool.cc:224] Handling call
0505 07:54:13.661679 (+175439us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:13.661869 (+ 190us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:13.661940 (+ 71us) catalog_manager.cc:2824] Locking tablets
0505 07:54:13.731899 (+ 69959us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:14.096639 (+364740us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:14.096761 (+ 122us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:14.531276 (+434515us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:14.531946 (+ 670us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:13.732832 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.732960 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.732968 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:13.732971 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:13.733076 (+ 105us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.733086 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.733090 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:13.733177 (+ 87us) tablet.cc:701] Row locks acquired
0505 07:54:13.733179 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:13.733208 (+ 29us) write_op.cc:270] Start()
0505 07:54:13.733236 (+ 28us) write_op.cc:276] Timestamp: P: 1777967653733159 usec, L: 0
0505 07:54:13.733239 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:13.733384 (+ 145us) log.cc:844] Serialized 725 byte log entry
0505 07:54:14.095742 (+362358us) op_driver.cc:464] REPLICATION: finished
0505 07:54:14.095942 (+ 200us) write_op.cc:301] APPLY: starting
0505 07:54:14.095962 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:14.096003 (+ 41us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:14.096028 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:14.096091 (+ 63us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:14.096124 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:14.096131 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:14.096212 (+ 81us) log.cc:844] Serialized 29 byte log entry
0505 07:54:14.096275 (+ 63us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.096299 (+ 24us) write_op.cc:454] Released schema lock
0505 07:54:14.096306 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":140,"apply.run_cpu_time_us":427,"apply.run_wall_time_us":427,"num_ops":2,"prepare.queue_time_us":266,"prepare.run_cpu_time_us":622,"prepare.run_wall_time_us":622,"replication_time_us":362454}]]}
W20260505 07:54:14.533550 717 leader_election.cc:343] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:14.535823 1350 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet f1ba93d114e746ef80697aa1401dd108 consensus state has no leader
I20260505 07:54:14.536026 1350 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 178) took 2276 ms. Trace:
I20260505 07:54:14.536115 1350 rpcz_store.cc:276] 0505 07:54:12.260004 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.483755 (+1223751us) service_pool.cc:224] Handling call
0505 07:54:13.667547 (+183792us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:13.667775 (+ 228us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:13.667939 (+ 164us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:13.667992 (+ 53us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:14.048872 (+380880us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:14.535965 (+487093us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:13.668542 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.668674 (+ 132us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.668682 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:13.668686 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:13.668817 (+ 131us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.668831 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.668861 (+ 30us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:13.668937 (+ 76us) tablet.cc:701] Row locks acquired
0505 07:54:13.668941 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:13.668978 (+ 37us) write_op.cc:270] Start()
0505 07:54:13.669018 (+ 40us) write_op.cc:276] Timestamp: P: 1777967653668932 usec, L: 0
0505 07:54:13.669024 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:13.669228 (+ 204us) log.cc:844] Serialized 405 byte log entry
0505 07:54:14.046119 (+376891us) op_driver.cc:464] REPLICATION: finished
0505 07:54:14.046288 (+ 169us) write_op.cc:301] APPLY: starting
0505 07:54:14.046298 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:14.046322 (+ 24us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:14.048322 (+ 2000us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:14.048395 (+ 73us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:14.048435 (+ 40us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:14.048446 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:14.048544 (+ 98us) log.cc:844] Serialized 23 byte log entry
0505 07:54:14.048635 (+ 91us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.048651 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:14.048656 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:14.048744 (+ 88us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.048751 (+ 7us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":132,"apply.run_cpu_time_us":508,"apply.run_wall_time_us":2442,"num_ops":1,"prepare.queue_time_us":56,"prepare.run_cpu_time_us":759,"prepare.run_wall_time_us":759,"replication_time_us":377081}]]}
I20260505 07:54:14.536993 1350 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6d0c4c13f95a4af2a119861b7923d94a" } new_table_name: "default.Table_3acf7c3eb13d473a9d952098c7524753" modify_external_catalogs: true
I20260505 07:54:14.539268 2833 raft_consensus.cc:515] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.539342 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 145) took 4728 ms. Trace:
I20260505 07:54:14.539431 2268 rpcz_store.cc:276] 0505 07:54:09.810396 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.813027 (+ 2631us) service_pool.cc:224] Handling call
0505 07:54:14.229165 (+4416138us) ts_tablet_manager.cc:1151] Deleting tablet a0f2f46d0e7840458cb992ce2a2dbea2
0505 07:54:14.229173 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.326780 (+ 97607us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.422043 (+ 95263us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.539325 (+117282us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4415990}
I20260505 07:54:14.539825 2268 tablet_replica.cc:333] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:14.539991 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 165) took 4991 ms. Trace:
I20260505 07:54:14.540061 2447 rpcz_store.cc:276] 0505 07:54:09.549000 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.549045 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:13.934690 (+4385645us) ts_tablet_manager.cc:1151] Deleting tablet 0db162dfad2448e8ac428b3d3fbe17f7
0505 07:54:13.934698 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.035500 (+100802us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.133561 (+ 98061us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.539958 (+406397us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4376437}
I20260505 07:54:14.540350 2447 tablet_replica.cc:333] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:14.540581 2447 raft_consensus.cc:2243] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:14.540982 2447 raft_consensus.cc:2272] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.541661 2447 ts_tablet_manager.cc:1916] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.543708 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 0db162dfad2448e8ac428b3d3fbe17f7 (table default.Table_73ba90aede024514abb230661a44a4ae [id=03a241143fae44a19531301524ef4adb]) successfully deleted
I20260505 07:54:14.544854 2268 raft_consensus.cc:2243] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.545006 2268 raft_consensus.cc:2272] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.545306 2280 raft_consensus.cc:515] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.545365 2872 raft_consensus.cc:515] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.545529 2268 ts_tablet_manager.cc:1916] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.545787 2280 leader_election.cc:290] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.546012 2856 raft_consensus.cc:493] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.546128 2856 raft_consensus.cc:515] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.546636 2856 leader_election.cc:290] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.546900 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "423c22891bed462d89e9ec51a079f0f5" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.547036 774 raft_consensus.cc:3060] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.548079 772 raft_consensus.cc:2468] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 2.
I20260505 07:54:14.548182 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 9f541eff399641dd81a0771186805493 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:13 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.548465 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet a0f2f46d0e7840458cb992ce2a2dbea2 (table default.Table_f8f7a899e8c94e75958bd11fed643bc6 [id=a3cea9a70f4147b7bf5b630980c94c27]) successfully deleted
I20260505 07:54:14.549582 774 raft_consensus.cc:2468] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:14.549605 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 9f541eff399641dd81a0771186805493 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:13 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.549934 988 leader_election.cc:304] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:14.550848 781 raft_consensus.cc:2468] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:14.551074 2670 raft_consensus.cc:2749] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:14.551151 2742 raft_consensus.cc:2804] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:14.551292 2742 raft_consensus.cc:697] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:14.551316 2670 raft_consensus.cc:493] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.551417 2670 raft_consensus.cc:515] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.551522 2742 consensus_queue.cc:237] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.551935 2670 leader_election.cc:290] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.552431 2650 raft_consensus.cc:493] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.552549 2650 raft_consensus.cc:515] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.553030 2650 leader_election.cc:290] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.553898 2405 raft_consensus.cc:493] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.554006 2405 raft_consensus.cc:515] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.554299 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72da22d790624b07bfca6f6b6301c925" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.554466 2405 leader_election.cc:290] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.554571 2405 consensus_queue.cc:1048] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.545789 2872 leader_election.cc:290] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.556321 854 leader_election.cc:304] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.556562 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3075e87e078f4d2aa1d329d18902354b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.556674 854 leader_election.cc:304] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.556689 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72da22d790624b07bfca6f6b6301c925" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.556749 907 raft_consensus.cc:2468] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:14.556979 2853 raft_consensus.cc:2804] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.557087 2853 raft_consensus.cc:697] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:14.557087 991 leader_election.cc:304] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:14.557229 718 leader_election.cc:304] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.557278 2853 consensus_queue.cc:237] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.557505 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9b7c6e9463342a5a73af12da4447f89" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.557615 916 raft_consensus.cc:3060] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:14.557694 854 leader_election.cc:343] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.557881 2752 raft_consensus.cc:2804] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
W20260505 07:54:14.557942 718 leader_election.cc:343] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 02412333633a4cdebe95a144fa6c363c
I20260505 07:54:14.557986 2752 raft_consensus.cc:697] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:14.558109 2872 raft_consensus.cc:2804] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.558200 2872 raft_consensus.cc:697] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:14.558184 2752 consensus_queue.cc:237] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.558390 2872 consensus_queue.cc:237] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.558673 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "429ccdb56f0b456483cf1debb5de0beb" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.558804 921 raft_consensus.cc:3060] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.559113 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3e076892234c1494b1a5c13cbfd2e2" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.559245 907 raft_consensus.cc:3060] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.559526 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6a5f823db6143319d743d363a77ea47" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.559701 781 raft_consensus.cc:2468] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:14.560140 916 raft_consensus.cc:2468] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.560320 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6a5f823db6143319d743d363a77ea47" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.560375 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3075e87e078f4d2aa1d329d18902354b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.560446 912 raft_consensus.cc:2468] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:14.560499 919 raft_consensus.cc:2468] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:14.560742 2650 raft_consensus.cc:2804] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.560806 2650 raft_consensus.cc:493] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.560855 2650 raft_consensus.cc:3060] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.560964 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3075e87e078f4d2aa1d329d18902354b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.561098 781 raft_consensus.cc:2468] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:14.561113 2276 ts_tablet_manager.cc:1929] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:14.561195 2276 log.cc:1199] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e29ba55a846548679cb3b86305d0d6ed
I20260505 07:54:14.561214 921 raft_consensus.cc:2468] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.561335 988 leader_election.cc:304] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.561524 2276 ts_tablet_manager.cc:1950] T e29ba55a846548679cb3b86305d0d6ed P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:14.561576 2405 raft_consensus.cc:2804] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.561657 2405 raft_consensus.cc:493] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.561736 2405 raft_consensus.cc:3060] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.562281 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 112) took 6780 ms. Trace:
I20260505 07:54:14.562348 2276 rpcz_store.cc:276] 0505 07:54:07.781788 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.799460 (+ 17672us) service_pool.cc:224] Handling call
0505 07:54:14.426170 (+6626710us) ts_tablet_manager.cc:1151] Deleting tablet e29ba55a846548679cb3b86305d0d6ed
0505 07:54:14.426179 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.521912 (+ 95733us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.561058 (+ 39146us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.562265 (+ 1207us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6626574}
I20260505 07:54:14.562623 2276 tablet_replica.cc:333] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:14.562778 2276 raft_consensus.cc:2243] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.562885 2276 raft_consensus.cc:2272] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.563344 2276 ts_tablet_manager.cc:1916] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.563962 2405 raft_consensus.cc:515] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.564570 2405 leader_election.cc:290] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.564949 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3075e87e078f4d2aa1d329d18902354b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.565276 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9b7c6e9463342a5a73af12da4447f89" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.565896 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6a5f823db6143319d743d363a77ea47" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.566022 781 raft_consensus.cc:3060] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.566138 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f3e076892234c1494b1a5c13cbfd2e2" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.567639 907 raft_consensus.cc:2468] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:14.567884 717 leader_election.cc:343] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 02412333633a4cdebe95a144fa6c363c
I20260505 07:54:14.567971 717 leader_election.cc:304] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.568143 2670 raft_consensus.cc:2749] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:14.569003 717 leader_election.cc:343] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:14.569186 717 leader_election.cc:343] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.569384 781 raft_consensus.cc:2468] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:14.569552 856 leader_election.cc:343] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.569670 856 leader_election.cc:304] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.570039 2856 raft_consensus.cc:2749] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.570281 2670 raft_consensus.cc:493] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.570381 2670 raft_consensus.cc:515] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.570817 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e29ba55a846548679cb3b86305d0d6ed (table default.Table_fd75ee1f117344169ccafd58dc18408f [id=5cd6a3d7d65d4a43943fa891ca480d7e]) successfully deleted
I20260505 07:54:14.570873 2670 leader_election.cc:290] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.571419 988 leader_election.cc:304] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.563488 2650 raft_consensus.cc:515] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.571638 2302 raft_consensus.cc:2804] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.571725 2302 raft_consensus.cc:697] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:14.571921 2302 consensus_queue.cc:237] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.572005 2650 leader_election.cc:290] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.572737 1048 raft_consensus.cc:2393] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:14.571436 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66be7b6c33a342d6890e459d90d40583" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.573788 1050 raft_consensus.cc:2468] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:14.574184 717 leader_election.cc:304] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.574412 2670 raft_consensus.cc:2804] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.574467 2670 raft_consensus.cc:493] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.574517 2670 raft_consensus.cc:3060] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.575881 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3075e87e078f4d2aa1d329d18902354b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.576521 2833 leader_election.cc:290] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:14.576761 991 leader_election.cc:343] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:54:14.577440 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 4258df74ea0841449846cdf1f6e00b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.577621 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f207a3ffbc94b01abd9e9d6837f5c81) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.581386 2670 raft_consensus.cc:515] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.581876 2670 leader_election.cc:290] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.582311 887 tablet_service.cc:1558] Processing DeleteTablet for tablet e29ba55a846548679cb3b86305d0d6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.582512 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.582646 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 02412333633a4cdebe95a144fa6c363c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:14.582603 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66be7b6c33a342d6890e459d90d40583" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.582751 1048 raft_consensus.cc:3060] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.583882 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6a5f823db6143319d743d363a77ea47" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.584018 907 raft_consensus.cc:3060] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.585196 1048 raft_consensus.cc:2468] T 66be7b6c33a342d6890e459d90d40583 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.585870 717 leader_election.cc:304] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.587066 2856 raft_consensus.cc:493] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.587183 2856 raft_consensus.cc:515] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.587291 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f842d8ac6b2142ffa69945c6087be34b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.587421 1050 raft_consensus.cc:3060] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.587805 2856 leader_election.cc:290] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.588469 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6546745e894545ff9c5f91bcc6e1a41e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:14.588939 854 leader_election.cc:343] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.589485 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6546745e894545ff9c5f91bcc6e1a41e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.590058 856 leader_election.cc:343] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.590175 856 leader_election.cc:304] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.590384 2856 raft_consensus.cc:2749] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:14.591321 718 consensus_peers.cc:597] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4258df74ea0841449846cdf1f6e00b9b. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:14.591372 919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66be7b6c33a342d6890e459d90d40583" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.591533 919 raft_consensus.cc:2468] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:14.592195 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66be7b6c33a342d6890e459d90d40583" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.592341 916 raft_consensus.cc:3060] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.591548 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f842d8ac6b2142ffa69945c6087be34b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.592782 912 raft_consensus.cc:3060] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.591658 2280 raft_consensus.cc:2804] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.594578 2280 raft_consensus.cc:697] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:14.594607 718 leader_election.cc:304] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.594866 2280 consensus_queue.cc:237] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.595119 2901 raft_consensus.cc:2804] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.595211 2901 raft_consensus.cc:697] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:14.595420 2901 consensus_queue.cc:237] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.591987 2833 raft_consensus.cc:493] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:14.595942 2833 raft_consensus.cc:515] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.596410 2833 leader_election.cc:290] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.592216 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3075e87e078f4d2aa1d329d18902354b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.596684 774 raft_consensus.cc:3060] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:14.596968 1358 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.2:57579 (request call id 156) took 3913 ms (client timeout 1000 ms). Trace:
W20260505 07:54:14.597079 1358 rpcz_store.cc:269] 0505 07:54:10.683152 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.735868 (+ 52716us) service_pool.cc:224] Handling call
0505 07:54:10.867914 (+132046us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d1358 00007fd65a467ca6 00007fd65a4679d5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:14.596930 (+3729016us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:11.292328 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:11.292443 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:11.292449 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:11.292451 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:11.292544 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:11.292558 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:11.292572 (+ 14us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:11.292632 (+ 60us) tablet.cc:701] Row locks acquired
0505 07:54:11.292634 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:11.292669 (+ 35us) write_op.cc:270] Start()
0505 07:54:11.292697 (+ 28us) write_op.cc:276] Timestamp: P: 1777967651292643 usec, L: 0
0505 07:54:11.292700 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:11.292863 (+ 163us) log.cc:844] Serialized 527 byte log entry
0505 07:54:11.296095 (+ 3232us) op_driver.cc:464] REPLICATION: finished
0505 07:54:11.296355 (+ 260us) write_op.cc:301] APPLY: starting
0505 07:54:11.296369 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:11.296412 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:11.296481 (+ 69us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:11.296529 (+ 48us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:11.296562 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:11.296570 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:11.296637 (+ 67us) log.cc:844] Serialized 23 byte log entry
0505 07:54:11.296703 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.296721 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:11.296726 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:11.296797 (+ 71us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:11.296800 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
0505 07:54:13.495796 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.495924 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.495932 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:13.495935 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:13.496054 (+ 119us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.496067 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.496073 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:13.496136 (+ 63us) tablet.cc:701] Row locks acquired
0505 07:54:13.496140 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:13.496177 (+ 37us) write_op.cc:270] Start()
0505 07:54:13.496211 (+ 34us) write_op.cc:276] Timestamp: P: 1777967653496149 usec, L: 0
0505 07:54:13.496216 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:13.496419 (+ 203us) log.cc:844] Serialized 356 byte log entry
0505 07:54:13.696328 (+199909us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.697050 (+ 722us) write_op.cc:301] APPLY: starting
0505 07:54:13.697075 (+ 25us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.697133 (+ 58us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.697173 (+ 40us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.697244 (+ 71us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.697289 (+ 45us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:13.697300 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:13.697415 (+ 115us) log.cc:844] Serialized 23 byte log entry
0505 07:54:13.697523 (+ 108us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.697548 (+ 25us) write_op.cc:454] Released schema lock
0505 07:54:13.697557 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:13.697703 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.697709 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":43246,"reported_tablets":248,"spinlock_wait_cycles":254087936,"child_traces":[["op",{"apply.queue_time_us":201,"apply.run_cpu_time_us":439,"apply.run_wall_time_us":439,"num_ops":1,"prepare.queue_time_us":39,"prepare.run_cpu_time_us":632,"prepare.run_wall_time_us":632,"raft.queue_time_us":57,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":253,"replication_time_us":3369}],["op",{"apply.queue_time_us":641,"apply.run_cpu_time_us":646,"apply.run_wall_time_us":646,"num_ops":1,"prepare.queue_time_us":59910,"prepare.run_cpu_time_us":710,"prepare.run_wall_time_us":710,"replication_time_us":200070}]]}
I20260505 07:54:14.598131 1358 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_063cc9f08ab54458b9e277ca65091b11" } modify_external_catalogs: true
I20260505 07:54:14.598300 1358 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_063cc9f08ab54458b9e277ca65091b11" } modify_external_catalogs: true
I20260505 07:54:14.599020 718 leader_election.cc:304] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.599326 718 leader_election.cc:304] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.599496 774 raft_consensus.cc:2468] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:14.600023 2268 ts_tablet_manager.cc:1929] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:14.600158 2268 log.cc:1199] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/32c008bd99224a7595cecc607e5496a9
I20260505 07:54:14.600445 2268 ts_tablet_manager.cc:1950] T 32c008bd99224a7595cecc607e5496a9 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:14.601019 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78a6470720984761a8606abeafbef992" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.601204 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 146) took 4787 ms. Trace:
I20260505 07:54:14.601282 2268 rpcz_store.cc:276] 0505 07:54:09.813292 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.813697 (+ 405us) service_pool.cc:224] Handling call
0505 07:54:14.539786 (+4726089us) ts_tablet_manager.cc:1151] Deleting tablet 32c008bd99224a7595cecc607e5496a9
0505 07:54:14.539794 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.574347 (+ 34553us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.599961 (+ 25614us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.601182 (+ 1221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4805,"tablet-delete.queue_time_us":4725941}
I20260505 07:54:14.601764 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 32c008bd99224a7595cecc607e5496a9 (table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]) successfully deleted
I20260505 07:54:14.602146 718 leader_election.cc:304] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.602996 2414 raft_consensus.cc:2804] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.603096 2414 raft_consensus.cc:697] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:14.603274 2414 consensus_queue.cc:237] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.602749 2833 raft_consensus.cc:2804] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.604396 2833 raft_consensus.cc:697] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:14.604677 2833 consensus_queue.cc:237] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.606302 2902 consensus_queue.cc:1048] T 3f869abaa31e414ea0625e475db08e64 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.610553 2853 raft_consensus.cc:493] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.610663 2853 raft_consensus.cc:515] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.611274 2853 leader_election.cc:290] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.611939 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a98243da14d43bc649bb19aed4f0c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.614660 2268 tablet_replica.cc:333] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:14.614750 1050 raft_consensus.cc:2468] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:14.615091 856 leader_election.cc:343] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.615271 717 leader_election.cc:304] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.615648 2670 raft_consensus.cc:2804] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.614874 2268 raft_consensus.cc:2243] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:14.615803 2670 raft_consensus.cc:697] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:14.615990 2268 raft_consensus.cc:2272] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.616039 2670 consensus_queue.cc:237] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.616536 2268 ts_tablet_manager.cc:1916] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.618556 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a98243da14d43bc649bb19aed4f0c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:14.619228 854 leader_election.cc:343] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:14.619310 1354 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3], current schema version=3): Not found: failed to reset TS proxy: No leader found: Tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 consensus state has no leader
I20260505 07:54:14.619347 854 leader_election.cc:304] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.619637 2853 raft_consensus.cc:2749] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.623685 2902 raft_consensus.cc:493] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
W20260505 07:54:14.623728 718 leader_election.cc:343] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 78a6470720984761a8606abeafbef992
I20260505 07:54:14.623802 2902 raft_consensus.cc:515] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.624336 2902 leader_election.cc:290] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.623806 718 leader_election.cc:304] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.626858 2902 raft_consensus.cc:2749] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:14.628171 1354 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.2:57579 (request call id 157) took 2830 ms (client timeout 1000 ms). Trace:
W20260505 07:54:14.628289 1354 rpcz_store.cc:269] 0505 07:54:11.797399 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.054733 (+257334us) service_pool.cc:224] Handling call
0505 07:54:14.628136 (+2573403us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:13.499717 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.499815 (+ 98us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.499821 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:13.499823 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:13.499904 (+ 81us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.499913 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.499916 (+ 3us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:13.499961 (+ 45us) tablet.cc:701] Row locks acquired
0505 07:54:13.499963 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:13.499992 (+ 29us) write_op.cc:270] Start()
0505 07:54:13.500019 (+ 27us) write_op.cc:276] Timestamp: P: 1777967653499965 usec, L: 0
0505 07:54:13.500022 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:13.500180 (+ 158us) log.cc:844] Serialized 356 byte log entry
0505 07:54:13.696416 (+196236us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.697794 (+ 1378us) write_op.cc:301] APPLY: starting
0505 07:54:13.697808 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.697843 (+ 35us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.697872 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.697917 (+ 45us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.697953 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:13.697964 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:13.698031 (+ 67us) log.cc:844] Serialized 23 byte log entry
0505 07:54:13.698077 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.698092 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:13.698100 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":67568,"reported_tablets":260,"child_traces":[["op",{"apply.queue_time_us":1338,"apply.run_cpu_time_us":367,"apply.run_wall_time_us":367,"num_ops":1,"prepare.queue_time_us":1331,"prepare.run_cpu_time_us":543,"prepare.run_wall_time_us":542,"replication_time_us":196372,"thread_start_us":19981,"threads_started":1}]]}
W20260505 07:54:14.628787 717 leader_election.cc:343] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 727cacd242844455915bd9963d2db19f
I20260505 07:54:14.629139 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727cacd242844455915bd9963d2db19f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.633248 907 raft_consensus.cc:2468] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:14.633816 2831 raft_consensus.cc:2764] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260505 07:54:14.634330 718 leader_election.cc:304] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.634640 2831 raft_consensus.cc:2749] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:14.634894 988 leader_election.cc:304] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:14.635650 916 raft_consensus.cc:2468] T 66be7b6c33a342d6890e459d90d40583 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.635798 2742 raft_consensus.cc:2804] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.635942 2742 raft_consensus.cc:697] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:14.636276 2742 consensus_queue.cc:237] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.638202 912 raft_consensus.cc:2468] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.640959 2447 ts_tablet_manager.cc:1929] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:14.641103 2447 log.cc:1199] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/067c32ecccb44bca8bf9cc3f6c229326
I20260505 07:54:14.641556 2447 ts_tablet_manager.cc:1950] T 067c32ecccb44bca8bf9cc3f6c229326 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:14.641891 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 166) took 5092 ms. Trace:
I20260505 07:54:14.641997 2447 rpcz_store.cc:276] 0505 07:54:09.549283 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.549353 (+ 70us) service_pool.cc:224] Handling call
0505 07:54:14.540305 (+4990952us) ts_tablet_manager.cc:1151] Deleting tablet 067c32ecccb44bca8bf9cc3f6c229326
0505 07:54:14.540317 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.579492 (+ 39175us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.640910 (+ 61418us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.641879 (+ 969us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4,"tablet-delete.queue_time_us":4982032}
I20260505 07:54:14.642405 2447 tablet_replica.cc:333] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:14.642580 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 067c32ecccb44bca8bf9cc3f6c229326 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]) successfully deleted
I20260505 07:54:14.642649 2447 raft_consensus.cc:2243] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.642777 2447 raft_consensus.cc:2272] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.643308 2447 ts_tablet_manager.cc:1916] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.651332 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_df175162fead408e9350bc9eb1de361c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:14.651945 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df175162fead408e9350bc9eb1de361c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:14.652442 2853 raft_consensus.cc:493] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.652557 2853 raft_consensus.cc:515] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.653067 2853 leader_election.cc:290] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.653744 2853 raft_consensus.cc:493] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.653887 2853 raft_consensus.cc:515] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.654459 2853 leader_election.cc:290] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.655078 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2706bc9b9e02414693062deff5cfe81b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.655514 856 leader_election.cc:343] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.655809 2853 raft_consensus.cc:493] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.655954 2853 raft_consensus.cc:515] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.656301 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2706bc9b9e02414693062deff5cfe81b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.656407 2853 leader_election.cc:290] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.657454 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e8f9f7b30f4912af916c15f9ae25d8" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.657490 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25072055f5534078adf5b5025614f951" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:14.657913 856 leader_election.cc:343] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.658268 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25072055f5534078adf5b5025614f951" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.658668 856 leader_election.cc:343] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:14.659230 854 leader_election.cc:343] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.659379 854 leader_election.cc:304] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.660112 2853 raft_consensus.cc:2749] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:14.660477 854 leader_election.cc:343] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.660610 854 leader_election.cc:304] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.660830 2853 raft_consensus.cc:2749] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.661252 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e8f9f7b30f4912af916c15f9ae25d8" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.661880 757 tablet_service.cc:1558] Processing DeleteTablet for tablet d896e290167b4d0081e8e8b831c8ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:14.662117 854 leader_election.cc:343] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.662262 854 leader_election.cc:304] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.662472 2853 raft_consensus.cc:2749] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.663167 2853 raft_consensus.cc:493] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.663264 2853 raft_consensus.cc:515] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.663750 2853 leader_election.cc:290] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.664593 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7887c3462f74740b314e3bec36e280d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:14.665005 856 leader_election.cc:343] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.665285 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7887c3462f74740b314e3bec36e280d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.666142 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 872362aecf6a44b59e586b1fa7637c8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.666828 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 01431ad112e8445487823efe8feb0dbe) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.667174 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet df1b62a30b2842b897a04c793bb5e7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.667495 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 297537a92684425d8508f7b3b4c9d379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.667943 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet 080b2984fcf2478a8def6540da63554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.668155 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet f200abef13ac42698a9b96fc8af82ff0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:14.669559 854 leader_election.cc:343] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.669721 854 leader_election.cc:304] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.669994 2853 raft_consensus.cc:2749] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:14.670404 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d6494590052459485059f5dce46f34f with delete_type TABLET_DATA_DELETED (replaced by tablet c822fafe72544ca99fde910c8e764755) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.671569 2650 consensus_queue.cc:1048] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.672379 1172 tablet_bootstrap.cc:492] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.672434 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 003cd4472b424aacbba8c5f89900e228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.672528 1172 ts_tablet_manager.cc:1403] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.211s user 0.002s sys 0.002s
I20260505 07:54:14.672911 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d83e9d4243e409794ffe8a342bce6b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.673290 757 tablet_service.cc:1558] Processing DeleteTablet for tablet e8bf49b787ef4de095448751464708a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.673457 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 472441bd29704be59aabb0f7a4964bb0 with delete_type TABLET_DATA_DELETED (replaced by tablet e7887c3462f74740b314e3bec36e280d) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.674113 1176 tablet_bootstrap.cc:492] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.674233 1176 ts_tablet_manager.cc:1403] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.194s user 0.003s sys 0.001s
I20260505 07:54:14.674289 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet e7887c3462f74740b314e3bec36e280d with delete_type TABLET_DATA_DELETED (replaced by tablet fba1d19f902a4a0a8b581135f6611129) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.674289 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.675647 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ad8a79245ad47738d079752e51aed2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.675763 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 with delete_type TABLET_DATA_DELETED (replaced by tablet d5083f98e8e14ccf99a19cdf2bc59018) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.676468 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.677570 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.677855 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.678172 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet e010f23c4df647ab863e03c3cfed7898 with delete_type TABLET_DATA_DELETED (replaced by tablet 09019fdf60374b11bf3427f21a55b5e0) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.678512 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.678776 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.679423 757 tablet_service.cc:1558] Processing DeleteTablet for tablet a7d8265c05b74b28959d0de41fad1fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.679631 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c4fc18f6faa488882d9bffb30793cdd) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.680392 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 727cacd242844455915bd9963d2db19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.680802 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e707143a4e64725aa66db05b24999a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.681931 757 tablet_service.cc:1558] Processing DeleteTablet for tablet bd581a6799a14a8dafb80d411e8936f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.683257 2276 ts_tablet_manager.cc:1929] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:14.683387 2276 log.cc:1199] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/3075e87e078f4d2aa1d329d18902354b
I20260505 07:54:14.683634 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 976359601ad74b149696dc7f555b550c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.683734 2276 ts_tablet_manager.cc:1950] T 3075e87e078f4d2aa1d329d18902354b P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:14.684064 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 7585f00119e14006990ed551667b4486 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:14.684537 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 113) took 6902 ms. Trace:
I20260505 07:54:14.684623 2276 rpcz_store.cc:276] 0505 07:54:07.781879 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.799628 (+ 17749us) service_pool.cc:224] Handling call
0505 07:54:14.562574 (+6762946us) ts_tablet_manager.cc:1151] Deleting tablet 3075e87e078f4d2aa1d329d18902354b
0505 07:54:14.562583 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.618847 (+ 56264us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.683195 (+ 64348us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.684501 (+ 1306us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6762835}
I20260505 07:54:14.684900 2276 tablet_replica.cc:333] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:14.685068 2276 raft_consensus.cc:2243] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.685181 2276 raft_consensus.cc:2272] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.686100 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 02412333633a4cdebe95a144fa6c363c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.686401 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 3075e87e078f4d2aa1d329d18902354b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.686630 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 3075e87e078f4d2aa1d329d18902354b (table default.Table_c1cf7717be07400aacdab563b4255450 [id=4d79fded98df46cc884cabdc3fba5d22]) successfully deleted
I20260505 07:54:14.686712 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet e29ba55a846548679cb3b86305d0d6ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.687079 2405 raft_consensus.cc:493] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:14.687233 2405 raft_consensus.cc:515] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.687441 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.687783 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet d3d11bd19f074bbab101448b45426b7a with delete_type TABLET_DATA_DELETED (replaced by tablet 76939b7978ee4f7ea0ad29312de2a31a) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.688315 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 4258df74ea0841449846cdf1f6e00b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.687079 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 46274a44a06b4306b5dac43c66c85a80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.689283 2405 leader_election.cc:290] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.691542 1176 raft_consensus.cc:359] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.691745 1176 raft_consensus.cc:385] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.691802 1176 raft_consensus.cc:740] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.694118 1172 raft_consensus.cc:359] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.694268 1172 raft_consensus.cc:385] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.694327 1172 raft_consensus.cc:740] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.694500 1172 consensus_queue.cc:260] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.695070 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.696623 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 3d6494590052459485059f5dce46f34f with delete_type TABLET_DATA_DELETED (replaced by tablet c822fafe72544ca99fde910c8e764755) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:14.697096 991 leader_election.cc:343] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: ff7ed926a3ff4e7993864de734ca3c50
I20260505 07:54:14.700287 2302 raft_consensus.cc:493] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
W20260505 07:54:14.700174 1355 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.1:36755 (request call id 145) took 3086 ms (client timeout 1000 ms). Trace:
I20260505 07:54:14.700457 2302 raft_consensus.cc:515] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.702509 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 32c008bd99224a7595cecc607e5496a9 with delete_type TABLET_DATA_DELETED (replaced by tablet a9494c3b6a99487596a030856b371b68) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.703296 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet a7fe9c7199cc4039ae877dc3d4aab17e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e08cf03ea0b495a94c6566ea6c8caad) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:14.700698 1355 rpcz_store.cc:269] 0505 07:54:11.613685 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.026996 (+413311us) service_pool.cc:224] Handling call
0505 07:54:14.700029 (+2673033us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.658971 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.659101 (+ 130us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.659108 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:12.659110 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:12.659219 (+ 109us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.659234 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.659242 (+ 8us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:12.659324 (+ 82us) tablet.cc:701] Row locks acquired
0505 07:54:12.659329 (+ 5us) write_op.cc:260] PREPARE: finished
0505 07:54:12.659371 (+ 42us) write_op.cc:270] Start()
0505 07:54:12.659407 (+ 36us) write_op.cc:276] Timestamp: P: 1777967652659260 usec, L: 0
0505 07:54:12.659411 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.659675 (+ 264us) log.cc:844] Serialized 527 byte log entry
0505 07:54:12.665060 (+ 5385us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.666234 (+ 1174us) write_op.cc:301] APPLY: starting
0505 07:54:12.666245 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.666270 (+ 25us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.666297 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.666333 (+ 36us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.666355 (+ 22us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:12.666361 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:12.666415 (+ 54us) log.cc:844] Serialized 23 byte log entry
0505 07:54:12.666452 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.666468 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:12.666476 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:12.666551 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.666557 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":64655,"reported_tablets":277,"child_traces":[["op",{"apply.queue_time_us":1146,"apply.run_cpu_time_us":303,"apply.run_wall_time_us":303,"num_ops":1,"prepare.queue_time_us":3781,"prepare.run_cpu_time_us":819,"prepare.run_wall_time_us":819,"raft.queue_time_us":85,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":351,"replication_time_us":5628}]]}
I20260505 07:54:14.704445 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 4783dd0b2fbe40219852d850fc0d9a82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.708455 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet a0f2f46d0e7840458cb992ce2a2dbea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.709553 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff7ed926a3ff4e7993864de734ca3c50" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.713207 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:14.715247 1172 ts_tablet_manager.cc:1434] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.043s user 0.001s sys 0.001s
I20260505 07:54:14.719650 1176 consensus_queue.cc:260] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.720214 1176 ts_tablet_manager.cc:1434] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.046s user 0.001s sys 0.000s
I20260505 07:54:14.720474 1176 tablet_bootstrap.cc:492] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:54:14.721268 1348 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.3:45737 (request call id 159) took 3677 ms (client timeout 1000 ms). Trace:
W20260505 07:54:14.721933 1348 rpcz_store.cc:269] 0505 07:54:11.043441 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.045029 (+ 1588us) service_pool.cc:224] Handling call
0505 07:54:11.139778 (+ 94749us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d1358 00007fd65a467ca6 00007fd65a4679d5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:14.721043 (+3581265us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:12.172088 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:12.172363 (+ 275us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:12.172370 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:12.172372 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:12.172847 (+ 475us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:12.172894 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:12.172900 (+ 6us) tablet.cc:685] Acquiring locks for 5 operations
0505 07:54:12.194643 (+ 21743us) tablet.cc:701] Row locks acquired
0505 07:54:12.194647 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:12.194682 (+ 35us) write_op.cc:270] Start()
0505 07:54:12.194717 (+ 35us) write_op.cc:276] Timestamp: P: 1777967652194616 usec, L: 0
0505 07:54:12.194721 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:12.194894 (+ 173us) log.cc:844] Serialized 2024 byte log entry
0505 07:54:12.300983 (+106089us) op_driver.cc:464] REPLICATION: finished
0505 07:54:12.314006 (+ 13023us) write_op.cc:301] APPLY: starting
0505 07:54:12.314023 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:12.314097 (+ 74us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:12.314131 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:12.314266 (+ 135us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:12.314300 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0505 07:54:12.314306 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:12.314383 (+ 77us) log.cc:844] Serialized 47 byte log entry
0505 07:54:12.314451 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:12.314474 (+ 23us) write_op.cc:454] Released schema lock
0505 07:54:12.314481 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:13.478921 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.479070 (+ 149us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.479077 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:13.479080 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:13.479183 (+ 103us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.479197 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.479204 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:13.479256 (+ 52us) tablet.cc:701] Row locks acquired
0505 07:54:13.479259 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:13.479295 (+ 36us) write_op.cc:270] Start()
0505 07:54:13.479325 (+ 30us) write_op.cc:276] Timestamp: P: 1777967653479270 usec, L: 0
0505 07:54:13.479331 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:13.479518 (+ 187us) log.cc:844] Serialized 356 byte log entry
0505 07:54:13.520121 (+ 40603us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.528614 (+ 8493us) write_op.cc:301] APPLY: starting
0505 07:54:13.528631 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.528684 (+ 53us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.528719 (+ 35us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.528777 (+ 58us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.528815 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:13.528823 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:13.528917 (+ 94us) log.cc:844] Serialized 23 byte log entry
0505 07:54:13.529769 (+ 852us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.529790 (+ 21us) write_op.cc:454] Released schema lock
0505 07:54:13.529800 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:13.538871 (+ 9071us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.538880 (+ 9us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":34383,"reported_tablets":276,"spinlock_wait_cycles":24891776,"child_traces":[["op",{"apply.queue_time_us":12940,"apply.run_cpu_time_us":549,"apply.run_wall_time_us":549,"num_ops":5,"prepare.queue_time_us":31543,"prepare.run_cpu_time_us":1296,"prepare.run_wall_time_us":32176,"raft.queue_time_us":57,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":198,"replication_time_us":106102,"thread_start_us":74275,"threads_started":2}],["op",{"apply.queue_time_us":8413,"apply.run_cpu_time_us":515,"apply.run_wall_time_us":10245,"num_ops":1,"prepare.queue_time_us":53997,"prepare.run_cpu_time_us":778,"prepare.run_wall_time_us":16844,"raft.queue_time_us":8609,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":329,"replication_time_us":40749,"thread_start_us":63747,"threads_started":2}]]}
I20260505 07:54:14.727936 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_fea44fc5aa9a4967a7e0ca87a0954aab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:14.728407 2302 leader_election.cc:290] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:14.729089 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fea44fc5aa9a4967a7e0ca87a0954aab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:14.730096 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 1838f89dc14a47f99f95f4cc5fdefd70 with delete_type TABLET_DATA_DELETED (replaced by tablet 4af9def97a454f3f9b63d637db42c110) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:14.733671 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 7585f00119e14006990ed551667b4486 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:14.735083 991 leader_election.cc:343] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 78a6470720984761a8606abeafbef992
I20260505 07:54:14.735946 1172 tablet_bootstrap.cc:492] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.737897 1172 tablet_bootstrap.cc:654] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.745476 2268 ts_tablet_manager.cc:1929] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:14.745579 2268 log.cc:1199] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/909306e1e5f34b7c8fbc0d86cafce531
I20260505 07:54:14.746448 2276 ts_tablet_manager.cc:1916] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.755095 2752 consensus_queue.cc:1048] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.755463 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78a6470720984761a8606abeafbef992" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.755710 772 raft_consensus.cc:2468] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:14.763852 2447 ts_tablet_manager.cc:1929] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:14.764007 2447 log.cc:1199] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/397d33e908af4a0bb8dde26d273be01a
I20260505 07:54:14.764561 2447 ts_tablet_manager.cc:1950] T 397d33e908af4a0bb8dde26d273be01a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:14.765832 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 168) took 5216 ms. Trace:
I20260505 07:54:14.765971 2447 rpcz_store.cc:276] 0505 07:54:09.549500 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.549549 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:14.642359 (+5092810us) ts_tablet_manager.cc:1151] Deleting tablet 397d33e908af4a0bb8dde26d273be01a
0505 07:54:14.642371 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.712961 (+ 70590us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.763773 (+ 50812us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.765816 (+ 2043us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5084032}
I20260505 07:54:14.766426 2447 tablet_replica.cc:333] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:14.767839 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 397d33e908af4a0bb8dde26d273be01a (table default.Table_499d356daab040e29ca2f5eb2941d790 [id=bf9eb924eb1e4586a6346bcf43be9a30]) successfully deleted
I20260505 07:54:14.771847 2853 raft_consensus.cc:493] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:14.771998 2853 raft_consensus.cc:515] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.772544 2853 leader_election.cc:290] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:14.773664 856 leader_election.cc:343] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 5ad8a79245ad47738d079752e51aed2b
I20260505 07:54:14.774947 1172 tablet_bootstrap.cc:492] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.775135 1172 ts_tablet_manager.cc:1403] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.039s user 0.002s sys 0.002s
W20260505 07:54:14.781894 856 consensus_peers.cc:597] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 472441bd29704be59aabb0f7a4964bb0. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:14.785087 854 leader_election.cc:343] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 5ad8a79245ad47738d079752e51aed2b
I20260505 07:54:14.785434 854 leader_election.cc:304] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.785995 2853 raft_consensus.cc:2749] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:14.771098 1181 tablet_bootstrap.cc:492] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:14.787552 1181 ts_tablet_manager.cc:1403] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.280s user 0.008s sys 0.003s
I20260505 07:54:14.788250 1181 raft_consensus.cc:359] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.788412 1181 raft_consensus.cc:385] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.788468 1181 raft_consensus.cc:740] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.788277 1172 raft_consensus.cc:359] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.788645 1181 consensus_queue.cc:260] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.788676 1172 raft_consensus.cc:385] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.788834 1172 raft_consensus.cc:740] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.789096 1181 ts_tablet_manager.cc:1434] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260505 07:54:14.789332 1172 consensus_queue.cc:260] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.789805 1181 tablet_bootstrap.cc:492] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:14.791128 1172 ts_tablet_manager.cc:1434] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.016s user 0.003s sys 0.000s
I20260505 07:54:14.794536 2674 consensus_queue.cc:1048] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.781670 988 leader_election.cc:304] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:14.796520 988 leader_election.cc:304] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:14.797145 2742 raft_consensus.cc:2749] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:14.797454 2742 raft_consensus.cc:2804] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:14.797509 2742 raft_consensus.cc:493] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:14.797570 2742 raft_consensus.cc:3060] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:14.800953 2742 raft_consensus.cc:515] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.801492 2742 leader_election.cc:290] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.801857 1181 tablet_bootstrap.cc:654] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:14.802208 991 leader_election.cc:343] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 78a6470720984761a8606abeafbef992
I20260505 07:54:14.803846 2447 raft_consensus.cc:2243] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.806433 2447 raft_consensus.cc:2272] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.808939 2901 raft_consensus.cc:493] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:14.809093 2901 raft_consensus.cc:515] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.813949 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff7ed926a3ff4e7993864de734ca3c50" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.814189 1048 raft_consensus.cc:2468] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:14.817034 717 leader_election.cc:304] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.817294 2674 raft_consensus.cc:2804] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:14.817384 2674 raft_consensus.cc:493] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:14.817456 2674 raft_consensus.cc:3060] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
W20260505 07:54:14.820417 718 leader_election.cc:343] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: ff7ed926a3ff4e7993864de734ca3c50
I20260505 07:54:14.820230 2674 raft_consensus.cc:515] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.821234 2674 leader_election.cc:290] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Requested vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.810071 2901 leader_election.cc:290] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.821734 1176 tablet_bootstrap.cc:654] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.824149 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff7ed926a3ff4e7993864de734ca3c50" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.824301 1050 raft_consensus.cc:3060] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:14.824906 2302 raft_consensus.cc:493] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.825191 2302 raft_consensus.cc:515] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.825718 2302 leader_election.cc:290] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.826864 1050 raft_consensus.cc:2468] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 2.
I20260505 07:54:14.827869 717 leader_election.cc:304] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.828639 2853 raft_consensus.cc:493] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.828697 2674 raft_consensus.cc:2804] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:14.828825 2674 raft_consensus.cc:697] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:14.828785 2853 raft_consensus.cc:515] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.829058 2674 consensus_queue.cc:237] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.829337 2853 leader_election.cc:290] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.830746 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080b2984fcf2478a8def6540da63554d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:14.830969 912 raft_consensus.cc:2468] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:14.831444 991 leader_election.cc:304] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:14.832041 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0d43d85fcd4d05abf10d415e8dafbe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.832209 767 raft_consensus.cc:2468] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:14.833395 2742 raft_consensus.cc:2804] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.833472 2742 raft_consensus.cc:493] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.833532 2742 raft_consensus.cc:3060] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.834684 854 leader_election.cc:304] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.836737 2856 raft_consensus.cc:2804] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.836846 2856 raft_consensus.cc:493] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.836936 2856 raft_consensus.cc:3060] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.840044 2856 raft_consensus.cc:515] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.840720 2856 leader_election.cc:290] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.841053 2742 raft_consensus.cc:515] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.846019 2742 leader_election.cc:290] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.846565 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080b2984fcf2478a8def6540da63554d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:14.846729 912 raft_consensus.cc:3060] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.847102 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0d43d85fcd4d05abf10d415e8dafbe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.847654 772 raft_consensus.cc:3060] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:14.844928 718 leader_election.cc:343] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: ff7ed926a3ff4e7993864de734ca3c50
I20260505 07:54:14.849229 912 raft_consensus.cc:2468] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:14.849905 991 leader_election.cc:304] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:14.850198 2742 raft_consensus.cc:2804] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.850636 2742 raft_consensus.cc:697] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:14.850915 2742 consensus_queue.cc:237] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.853204 2302 raft_consensus.cc:493] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:14.853349 2302 raft_consensus.cc:515] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:14.854166 2302 leader_election.cc:290] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:14.856686 1172 tablet_bootstrap.cc:492] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.858071 2268 ts_tablet_manager.cc:1950] T 909306e1e5f34b7c8fbc0d86cafce531 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:14.858831 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 147) took 5044 ms. Trace:
I20260505 07:54:14.858971 2268 rpcz_store.cc:276] 0505 07:54:09.813957 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.814310 (+ 353us) service_pool.cc:224] Handling call
0505 07:54:14.614610 (+4800300us) ts_tablet_manager.cc:1151] Deleting tablet 909306e1e5f34b7c8fbc0d86cafce531
0505 07:54:14.614620 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.742498 (+127878us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.745429 (+ 2931us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.858814 (+113385us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4800176}
I20260505 07:54:14.859412 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 148) took 5044 ms. Trace:
I20260505 07:54:14.859540 2268 rpcz_store.cc:276] 0505 07:54:09.814589 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.815115 (+ 526us) service_pool.cc:224] Handling call
0505 07:54:14.859300 (+5044185us) ts_tablet_manager.cc:1151] Deleting tablet e82e9ae417784b7e83f4843b129d0580
0505 07:54:14.859310 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.859397 (+ 87us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5044047}
I20260505 07:54:14.859925 2268 tablet_replica.cc:333] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:14.860165 2268 raft_consensus.cc:2243] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.860354 2268 raft_consensus.cc:2272] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:14.860836 991 leader_election.cc:343] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 23fa401bb4634022bca3c6b23421e4d5
I20260505 07:54:14.860859 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 909306e1e5f34b7c8fbc0d86cafce531 (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]) successfully deleted
W20260505 07:54:14.861347 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e82e9ae417784b7e83f4843b129d0580 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e82e9ae417784b7e83f4843b129d0580 already in progress: creating tablet
I20260505 07:54:14.863628 2447 ts_tablet_manager.cc:1916] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.868673 1172 tablet_bootstrap.cc:654] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.871300 1348 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 184) took 2286 ms. Trace:
I20260505 07:54:14.871829 2856 raft_consensus.cc:493] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.871436 1348 rpcz_store.cc:276] 0505 07:54:12.585036 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.725917 (+2140881us) service_pool.cc:224] Handling call
0505 07:54:14.727860 (+ 1943us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d6b13 00007fd65a46a7c6 00007fd65a46a4f5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:14.752122 (+ 24262us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:14.752559 (+ 437us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:14.779693 (+ 27134us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:14.871146 (+ 91453us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:14.871256 (+ 110us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:14.773414 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:14.773514 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:14.773520 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:14.773524 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:14.773599 (+ 75us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:14.773607 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:14.773611 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:14.773705 (+ 94us) tablet.cc:701] Row locks acquired
0505 07:54:14.773707 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:14.773734 (+ 27us) write_op.cc:270] Start()
0505 07:54:14.773813 (+ 79us) write_op.cc:276] Timestamp: P: 1777967654773669 usec, L: 0
0505 07:54:14.773816 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:14.773955 (+ 139us) log.cc:844] Serialized 433 byte log entry
0505 07:54:14.777815 (+ 3860us) op_driver.cc:464] REPLICATION: finished
0505 07:54:14.779287 (+ 1472us) write_op.cc:301] APPLY: starting
0505 07:54:14.779297 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:14.779328 (+ 31us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:14.779353 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:14.779439 (+ 86us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:14.779467 (+ 28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:14.779476 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:14.779530 (+ 54us) log.cc:844] Serialized 29 byte log entry
0505 07:54:14.779605 (+ 75us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.779619 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:14.779624 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":3421056,"child_traces":[["op",{"apply.queue_time_us":1442,"apply.run_cpu_time_us":394,"apply.run_wall_time_us":496,"num_ops":2,"prepare.queue_time_us":20415,"prepare.run_cpu_time_us":576,"prepare.run_wall_time_us":624,"raft.queue_time_us":291,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"replication_time_us":3982,"thread_start_us":26995,"threads_started":1}]]}
I20260505 07:54:14.871958 2856 raft_consensus.cc:515] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.872476 2856 leader_election.cc:290] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:14.875625 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 78685.3us
I20260505 07:54:14.876606 757 tablet_service.cc:1511] Processing CreateTablet for tablet 0ffcea5808c34be595bd870c0b0f6e10 (DEFAULT_TABLE table=default.Table_f39ab9af40ff46588e5ea492e2e094c0 [id=64904669614d4b51a1ffe3376c05bc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.876972 757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ffcea5808c34be595bd870c0b0f6e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.876417 1181 tablet_bootstrap.cc:492] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:14.878012 1181 ts_tablet_manager.cc:1403] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.088s user 0.004s sys 0.006s
I20260505 07:54:14.879714 1181 raft_consensus.cc:359] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.879964 1354 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 183) took 2295 ms. Trace:
I20260505 07:54:14.880105 1354 rpcz_store.cc:276] 0505 07:54:12.584920 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.628828 (+2043908us) service_pool.cc:224] Handling call
0505 07:54:14.751115 (+122287us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:14.751683 (+ 568us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:14.779642 (+ 27959us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:14.879794 (+100152us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:14.879905 (+ 111us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:14.770132 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:14.770224 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:14.770228 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:14.770230 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:14.770307 (+ 77us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:14.770319 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:14.770323 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:14.770384 (+ 61us) tablet.cc:701] Row locks acquired
0505 07:54:14.770386 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:14.770415 (+ 29us) write_op.cc:270] Start()
0505 07:54:14.770446 (+ 31us) write_op.cc:276] Timestamp: P: 1777967654770348 usec, L: 0
0505 07:54:14.770449 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:14.773017 (+ 2568us) log.cc:844] Serialized 433 byte log entry
0505 07:54:14.773324 (+ 307us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65bae8f22 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac
0505 07:54:14.777759 (+ 4435us) op_driver.cc:464] REPLICATION: finished
0505 07:54:14.778799 (+ 1040us) write_op.cc:301] APPLY: starting
0505 07:54:14.778811 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:14.778843 (+ 32us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:14.778846 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:14.778978 (+ 132us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:14.779011 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:14.779019 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:14.779080 (+ 61us) log.cc:844] Serialized 29 byte log entry
0505 07:54:14.779135 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.779148 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:14.779153 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:14.779220 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.779225 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1001,"apply.run_cpu_time_us":412,"apply.run_wall_time_us":411,"num_ops":2,"prepare.queue_time_us":18014,"prepare.run_cpu_time_us":806,"prepare.run_wall_time_us":3253,"raft.queue_time_us":721,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":203,"replication_time_us":7285,"spinlock_wait_cycles":5565312}]]}
I20260505 07:54:14.880824 751 tablet_service.cc:1511] Processing CreateTablet for tablet 92d7da89f9a549f281493386863bd462 (DEFAULT_TABLE table=default.Table_be7562a92105457caa8d9ad5d34012b6 [id=5db98369e1184a26ba34f1e1f269d0a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.880909 760 tablet_service.cc:1511] Processing CreateTablet for tablet 27023732a54d41f3afd0030e773fb548 (DEFAULT_TABLE table=default.Table_101c994cc1964e8a88f0bdb4fc08b1b9 [id=2476f6c17add4031841ef5f0567d4541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.881134 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92d7da89f9a549f281493386863bd462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.881031 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_8f7522cabddb40e0b938edb0f45626c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:14.881647 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27023732a54d41f3afd0030e773fb548. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:14.882018 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8f7522cabddb40e0b938edb0f45626c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:14.883827 758 tablet_service.cc:1511] Processing CreateTablet for tablet 71ab9c7e57de4cd1a666c86cc461adad (DEFAULT_TABLE table=default.Table_fe77d748257e4d848042a96b91636eb2 [id=99366a3c94b54ba083d7b706756104e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.884130 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ab9c7e57de4cd1a666c86cc461adad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.886710 749 tablet_service.cc:1511] Processing CreateTablet for tablet 48afb0b97fec4cb4881f39dfb0b1bba8 (DEFAULT_TABLE table=default.Table_3f9cf31daaba4d4f95774e2628b068cb [id=4dea69169f6a4002929c29fbd0b006b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.887027 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48afb0b97fec4cb4881f39dfb0b1bba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.889153 899 tablet_service.cc:1511] Processing CreateTablet for tablet 92d7da89f9a549f281493386863bd462 (DEFAULT_TABLE table=default.Table_be7562a92105457caa8d9ad5d34012b6 [id=5db98369e1184a26ba34f1e1f269d0a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.889526 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92d7da89f9a549f281493386863bd462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.889840 887 tablet_service.cc:1511] Processing CreateTablet for tablet 0ffcea5808c34be595bd870c0b0f6e10 (DEFAULT_TABLE table=default.Table_f39ab9af40ff46588e5ea492e2e094c0 [id=64904669614d4b51a1ffe3376c05bc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.890152 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ffcea5808c34be595bd870c0b0f6e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.892222 744 tablet_service.cc:1511] Processing CreateTablet for tablet 14ad3c4ad3c143d4862a9c33f1b46e49 (DEFAULT_TABLE table=default.Table_6e7a27e38f8a4feb82d93ea27356ba60 [id=19234a77076744f887c8c7865c3dc36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.892519 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ad3c4ad3c143d4862a9c33f1b46e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.892763 892 tablet_service.cc:1511] Processing CreateTablet for tablet 27023732a54d41f3afd0030e773fb548 (DEFAULT_TABLE table=default.Table_101c994cc1964e8a88f0bdb4fc08b1b9 [id=2476f6c17add4031841ef5f0567d4541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.893061 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27023732a54d41f3afd0030e773fb548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.895974 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82e9ae417784b7e83f4843b129d0580" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.896118 885 tablet_service.cc:1511] Processing CreateTablet for tablet 48afb0b97fec4cb4881f39dfb0b1bba8 (DEFAULT_TABLE table=default.Table_3f9cf31daaba4d4f95774e2628b068cb [id=4dea69169f6a4002929c29fbd0b006b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.896438 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48afb0b97fec4cb4881f39dfb0b1bba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.897009 754 tablet_service.cc:1511] Processing CreateTablet for tablet 75de7522fa534140822ead8be4dc2b2d (DEFAULT_TABLE table=default.Table_f38a3721f0774a7bbc71a46b8107ee83 [id=a28a37343b9d4b718ed921c66a2d67e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.897341 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75de7522fa534140822ead8be4dc2b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.897575 758 tablet_service.cc:1511] Processing CreateTablet for tablet d5b391d72d784c4db3457e464ad5ee93 (DEFAULT_TABLE table=default.Table_d4c7087457154e89b20abe713355e422 [id=7099b9d0095f4bf880cff569eeeb5d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.897890 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b391d72d784c4db3457e464ad5ee93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.901605 2276 ts_tablet_manager.cc:1929] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:14.901726 2276 log.cc:1199] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/27bacc97d7534284970f218cf68ac75b
I20260505 07:54:14.902072 2276 ts_tablet_manager.cc:1950] T 27bacc97d7534284970f218cf68ac75b P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:14.902899 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 114) took 7120 ms. Trace:
I20260505 07:54:14.903017 2276 rpcz_store.cc:276] 0505 07:54:07.781938 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.799753 (+ 17815us) service_pool.cc:224] Handling call
0505 07:54:14.684860 (+6885107us) ts_tablet_manager.cc:1151] Deleting tablet 27bacc97d7534284970f218cf68ac75b
0505 07:54:14.684868 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.809831 (+124963us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.901507 (+ 91676us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.902851 (+ 1344us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6884979}
I20260505 07:54:14.903398 2276 tablet_replica.cc:333] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:14.903699 2856 raft_consensus.cc:493] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:14.903865 2856 raft_consensus.cc:515] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.911823 894 tablet_service.cc:1511] Processing CreateTablet for tablet 71ab9c7e57de4cd1a666c86cc461adad (DEFAULT_TABLE table=default.Table_fe77d748257e4d848042a96b91636eb2 [id=99366a3c94b54ba083d7b706756104e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.912190 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ab9c7e57de4cd1a666c86cc461adad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.910882 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0d43d85fcd4d05abf10d415e8dafbe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:14.912724 1181 raft_consensus.cc:385] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.912789 1181 raft_consensus.cc:740] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.912766 1021 tablet_service.cc:1511] Processing CreateTablet for tablet 92d7da89f9a549f281493386863bd462 (DEFAULT_TABLE table=default.Table_be7562a92105457caa8d9ad5d34012b6 [id=5db98369e1184a26ba34f1e1f269d0a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.912943 1181 consensus_queue.cc:260] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.913086 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92d7da89f9a549f281493386863bd462. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.913427 1181 ts_tablet_manager.cc:1434] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.034s user 0.001s sys 0.000s
I20260505 07:54:14.913988 751 tablet_service.cc:1511] Processing CreateTablet for tablet b696332560704328b396a2be587ebc1b (DEFAULT_TABLE table=default.Table_1000a6ae9ea548fea1b484b91e275b82 [id=5e10b83b0fc049eda789e1df440b044a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.914189 1181 tablet_bootstrap.cc:492] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:14.860852 2268 ts_tablet_manager.cc:1916] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.911871 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0d43d85fcd4d05abf10d415e8dafbe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:14.912307 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 0ffcea5808c34be595bd870c0b0f6e10 (DEFAULT_TABLE table=default.Table_f39ab9af40ff46588e5ea492e2e094c0 [id=64904669614d4b51a1ffe3376c05bc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.920699 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ffcea5808c34be595bd870c0b0f6e10. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:14.922036 854 leader_election.cc:343] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:14.922844 895 tablet_service.cc:1511] Processing CreateTablet for tablet c6b83ecd061f4a008cd3467521a1c5e0 (DEFAULT_TABLE table=default.Table_ab051f01f8054d1b91cbe477cef39adc [id=114ba761f7b846a2a86027b62071eb13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.923017 900 tablet_service.cc:1511] Processing CreateTablet for tablet 75de7522fa534140822ead8be4dc2b2d (DEFAULT_TABLE table=default.Table_f38a3721f0774a7bbc71a46b8107ee83 [id=a28a37343b9d4b718ed921c66a2d67e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.923177 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b83ecd061f4a008cd3467521a1c5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.923448 896 tablet_service.cc:1511] Processing CreateTablet for tablet b696332560704328b396a2be587ebc1b (DEFAULT_TABLE table=default.Table_1000a6ae9ea548fea1b484b91e275b82 [id=5e10b83b0fc049eda789e1df440b044a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.923800 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b696332560704328b396a2be587ebc1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.926280 901 tablet_service.cc:1511] Processing CreateTablet for tablet 06d87dc359cb48c2b5a6f5008f33427e (DEFAULT_TABLE table=default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.926401 749 tablet_service.cc:1511] Processing CreateTablet for tablet c6b83ecd061f4a008cd3467521a1c5e0 (DEFAULT_TABLE table=default.Table_ab051f01f8054d1b91cbe477cef39adc [id=114ba761f7b846a2a86027b62071eb13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.926553 901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d87dc359cb48c2b5a6f5008f33427e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.926733 760 tablet_service.cc:1511] Processing CreateTablet for tablet 06d87dc359cb48c2b5a6f5008f33427e (DEFAULT_TABLE table=default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.927345 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080b2984fcf2478a8def6540da63554d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.927534 775 raft_consensus.cc:3060] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.922427 899 tablet_service.cc:1511] Processing CreateTablet for tablet 14ad3c4ad3c143d4862a9c33f1b46e49 (DEFAULT_TABLE table=default.Table_6e7a27e38f8a4feb82d93ea27356ba60 [id=19234a77076744f887c8c7865c3dc36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.927891 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ad3c4ad3c143d4862a9c33f1b46e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.930127 775 raft_consensus.cc:2468] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:14.930327 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080b2984fcf2478a8def6540da63554d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.930519 777 raft_consensus.cc:2376] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1. Re-sending same reply.
I20260505 07:54:14.922844 888 tablet_service.cc:1511] Processing CreateTablet for tablet d5b391d72d784c4db3457e464ad5ee93 (DEFAULT_TABLE table=default.Table_d4c7087457154e89b20abe713355e422 [id=7099b9d0095f4bf880cff569eeeb5d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.930908 888 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b391d72d784c4db3457e464ad5ee93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.931005 1181 tablet_bootstrap.cc:654] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:14.931658 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b696332560704328b396a2be587ebc1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.932627 744 tablet_service.cc:1511] Processing CreateTablet for tablet 89d3e08134704859abc44257f484b8cd (DEFAULT_TABLE table=default.Table_02afe59354994bffb24b190e9a7abf85 [id=7579207045ec4220b966af9016a8f050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.932981 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d3e08134704859abc44257f484b8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.934381 2670 consensus_queue.cc:1048] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:14.935478 772 raft_consensus.cc:2468] T ed0d43d85fcd4d05abf10d415e8dafbe P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:14.937575 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b83ecd061f4a008cd3467521a1c5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.938874 900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75de7522fa534140822ead8be4dc2b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.940655 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78a6470720984761a8606abeafbef992" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.940810 774 raft_consensus.cc:3060] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:14.942095 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 71ab9c7e57de4cd1a666c86cc461adad (DEFAULT_TABLE table=default.Table_fe77d748257e4d848042a96b91636eb2 [id=99366a3c94b54ba083d7b706756104e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.942389 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71ab9c7e57de4cd1a666c86cc461adad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.943836 1021 tablet_service.cc:1511] Processing CreateTablet for tablet 27023732a54d41f3afd0030e773fb548 (DEFAULT_TABLE table=default.Table_101c994cc1964e8a88f0bdb4fc08b1b9 [id=2476f6c17add4031841ef5f0567d4541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.944128 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27023732a54d41f3afd0030e773fb548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.945122 1172 tablet_bootstrap.cc:492] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:14.945237 1172 ts_tablet_manager.cc:1403] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.095s user 0.004s sys 0.000s
I20260505 07:54:14.945924 1172 raft_consensus.cc:359] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.946069 1172 raft_consensus.cc:385] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.946115 1172 raft_consensus.cc:740] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.946302 1172 consensus_queue.cc:260] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.946542 2853 raft_consensus.cc:493] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:14.946677 2853 raft_consensus.cc:515] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.946774 1172 ts_tablet_manager.cc:1434] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:14.947211 2853 leader_election.cc:290] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.947490 854 leader_election.cc:304] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.947723 2853 raft_consensus.cc:2804] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:14.947829 2853 raft_consensus.cc:697] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:14.948027 2853 consensus_queue.cc:237] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.951764 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 48afb0b97fec4cb4881f39dfb0b1bba8 (DEFAULT_TABLE table=default.Table_3f9cf31daaba4d4f95774e2628b068cb [id=4dea69169f6a4002929c29fbd0b006b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.952060 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48afb0b97fec4cb4881f39dfb0b1bba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.952128 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 27bacc97d7534284970f218cf68ac75b (table default.Table_db40b13ba59549c6bae350382395bceb [id=db770342dce84ba1ab28d029ddb65d78]) successfully deleted
I20260505 07:54:14.952646 2749 raft_consensus.cc:493] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:14.952754 2749 raft_consensus.cc:515] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.953220 2749 leader_election.cc:290] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.953747 2268 ts_tablet_manager.cc:1929] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:14.953830 2268 log.cc:1199] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/8b406c4c6ba741bc85eef54c6f576459
I20260505 07:54:14.954077 2268 ts_tablet_manager.cc:1950] T 8b406c4c6ba741bc85eef54c6f576459 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:14.954679 899 tablet_service.cc:1511] Processing CreateTablet for tablet 89d3e08134704859abc44257f484b8cd (DEFAULT_TABLE table=default.Table_02afe59354994bffb24b190e9a7abf85 [id=7579207045ec4220b966af9016a8f050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.955289 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 149) took 5139 ms. Trace:
I20260505 07:54:14.955360 2268 rpcz_store.cc:276] 0505 07:54:09.815393 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.816671 (+ 1278us) service_pool.cc:224] Handling call
0505 07:54:14.859865 (+5043194us) ts_tablet_manager.cc:1151] Deleting tablet 8b406c4c6ba741bc85eef54c6f576459
0505 07:54:14.859874 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.943758 (+ 83884us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.953699 (+ 9941us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.955273 (+ 1574us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5043060}
I20260505 07:54:14.955655 2268 tablet_replica.cc:333] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:14.955819 2268 raft_consensus.cc:2243] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.955937 2268 raft_consensus.cc:2272] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.956377 2268 ts_tablet_manager.cc:1916] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.956620 895 tablet_service.cc:1511] Processing CreateTablet for tablet f845a4a0795e491c937c56e208c7d173 (DEFAULT_TABLE table=default.Table_ff4ec4dc16ec486d939b9b6e233ceaad [id=e055cb39dba9491eb64e346f89e45602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.957688 2276 raft_consensus.cc:2243] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:14.957911 2276 raft_consensus.cc:2272] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:14.958251 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d3e08134704859abc44257f484b8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.958367 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 14ad3c4ad3c143d4862a9c33f1b46e49 (DEFAULT_TABLE table=default.Table_6e7a27e38f8a4feb82d93ea27356ba60 [id=19234a77076744f887c8c7865c3dc36f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.958657 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14ad3c4ad3c143d4862a9c33f1b46e49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.958904 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f845a4a0795e491c937c56e208c7d173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.962338 1172 tablet_bootstrap.cc:492] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:14.963418 2276 ts_tablet_manager.cc:1916] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:14.964113 1023 tablet_service.cc:1511] Processing CreateTablet for tablet d5b391d72d784c4db3457e464ad5ee93 (DEFAULT_TABLE table=default.Table_d4c7087457154e89b20abe713355e422 [id=7099b9d0095f4bf880cff569eeeb5d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.952261 2674 raft_consensus.cc:493] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:14.964489 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b391d72d784c4db3457e464ad5ee93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.964468 2674 raft_consensus.cc:515] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.965119 2674 leader_election.cc:290] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.954716 751 tablet_service.cc:1511] Processing CreateTablet for tablet f845a4a0795e491c937c56e208c7d173 (DEFAULT_TABLE table=default.Table_ff4ec4dc16ec486d939b9b6e233ceaad [id=e055cb39dba9491eb64e346f89e45602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.944339 2447 ts_tablet_manager.cc:1929] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
W20260505 07:54:14.965760 717 leader_election.cc:343] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 46274a44a06b4306b5dac43c66c85a80
I20260505 07:54:14.966060 2447 log.cc:1199] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/43e4c92b7c3140fdb2d6a290aaf0ad55
W20260505 07:54:14.967721 854 leader_election.cc:343] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 1602ccbae24544c4a6cf14ea073880b4
I20260505 07:54:14.967551 2447 ts_tablet_manager.cc:1950] T 43e4c92b7c3140fdb2d6a290aaf0ad55 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:14.975839 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d87dc359cb48c2b5a6f5008f33427e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.977178 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727cacd242844455915bd9963d2db19f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.977409 772 raft_consensus.cc:2468] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:14.978746 2856 leader_election.cc:290] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.981606 854 leader_election.cc:304] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
W20260505 07:54:14.982359 718 leader_election.cc:343] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 46274a44a06b4306b5dac43c66c85a80
I20260505 07:54:14.982503 718 leader_election.cc:304] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:14.982863 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 75de7522fa534140822ead8be4dc2b2d (DEFAULT_TABLE table=default.Table_f38a3721f0774a7bbc71a46b8107ee83 [id=a28a37343b9d4b718ed921c66a2d67e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.983023 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "139d5a2ab9594f6ea6042c1c4ba8a46a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:14.983182 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75de7522fa534140822ead8be4dc2b2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.983239 781 raft_consensus.cc:2468] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:14.983268 2674 raft_consensus.cc:2749] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:14.983842 854 leader_election.cc:304] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.984076 2752 raft_consensus.cc:2804] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:14.984144 2752 raft_consensus.cc:493] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:14.984226 2752 raft_consensus.cc:3060] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:14.985764 2856 raft_consensus.cc:2804] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:14.986109 2856 raft_consensus.cc:493] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:14.986521 2856 raft_consensus.cc:3060] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:14.988083 1031 tablet_service.cc:1511] Processing CreateTablet for tablet b696332560704328b396a2be587ebc1b (DEFAULT_TABLE table=default.Table_1000a6ae9ea548fea1b484b91e275b82 [id=5e10b83b0fc049eda789e1df440b044a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.988840 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b696332560704328b396a2be587ebc1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.957942 1176 tablet_bootstrap.cc:492] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:14.989261 1176 ts_tablet_manager.cc:1403] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.269s user 0.004s sys 0.000s
I20260505 07:54:14.989971 1176 raft_consensus.cc:359] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.990154 1176 raft_consensus.cc:385] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:14.990208 1176 raft_consensus.cc:740] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:14.986613 2752 raft_consensus.cc:515] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.991470 2752 leader_election.cc:290] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:14.991828 1020 tablet_service.cc:1511] Processing CreateTablet for tablet c6b83ecd061f4a008cd3467521a1c5e0 (DEFAULT_TABLE table=default.Table_ab051f01f8054d1b91cbe477cef39adc [id=114ba761f7b846a2a86027b62071eb13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.992097 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6b83ecd061f4a008cd3467521a1c5e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.992043 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "727cacd242844455915bd9963d2db19f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:14.992201 781 raft_consensus.cc:3060] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:14.995136 781 raft_consensus.cc:2468] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 2.
I20260505 07:54:14.995464 854 leader_election.cc:304] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:14.995735 2752 raft_consensus.cc:2804] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:14.995918 2752 raft_consensus.cc:697] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:14.995923 1019 tablet_service.cc:1511] Processing CreateTablet for tablet 06d87dc359cb48c2b5a6f5008f33427e (DEFAULT_TABLE table=default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:14.996222 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d87dc359cb48c2b5a6f5008f33427e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:14.996140 2752 consensus_queue.cc:237] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:14.997179 774 raft_consensus.cc:2468] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 2.
I20260505 07:54:14.999820 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 89d3e08134704859abc44257f484b8cd (DEFAULT_TABLE table=default.Table_02afe59354994bffb24b190e9a7abf85 [id=7579207045ec4220b966af9016a8f050]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:15.000097 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89d3e08134704859abc44257f484b8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:15.000351 1172 tablet_bootstrap.cc:654] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.003758 1176 consensus_queue.cc:260] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.004153 1027 tablet_service.cc:1511] Processing CreateTablet for tablet f845a4a0795e491c937c56e208c7d173 (DEFAULT_TABLE table=default.Table_ff4ec4dc16ec486d939b9b6e233ceaad [id=e055cb39dba9491eb64e346f89e45602]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:15.004436 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f845a4a0795e491c937c56e208c7d173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:15.007201 2856 raft_consensus.cc:515] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.010140 2276 ts_tablet_manager.cc:1929] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:15.010859 2276 log.cc:1199] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/6546745e894545ff9c5f91bcc6e1a41e
I20260505 07:54:15.013617 2856 leader_election.cc:290] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.013852 2276 ts_tablet_manager.cc:1950] T 6546745e894545ff9c5f91bcc6e1a41e P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.016963 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82e9ae417784b7e83f4843b129d0580" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.017670 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "139d5a2ab9594f6ea6042c1c4ba8a46a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.017932 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "139d5a2ab9594f6ea6042c1c4ba8a46a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:15.019052 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 115) took 7236 ms. Trace:
I20260505 07:54:15.019361 1176 ts_tablet_manager.cc:1434] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.030s user 0.001s sys 0.001s
I20260505 07:54:15.019533 2276 rpcz_store.cc:276] 0505 07:54:07.782003 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.799893 (+ 17890us) service_pool.cc:224] Handling call
0505 07:54:14.903333 (+7103440us) ts_tablet_manager.cc:1151] Deleting tablet 6546745e894545ff9c5f91bcc6e1a41e
0505 07:54:14.903344 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.986451 (+ 83107us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.009865 (+ 23414us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.018865 (+ 9000us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7103325}
I20260505 07:54:15.021200 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 6546745e894545ff9c5f91bcc6e1a41e (table default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]) successfully deleted
I20260505 07:54:15.021843 2276 tablet_replica.cc:333] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.022790 2276 raft_consensus.cc:2243] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.023135 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:15.024138 2872 raft_consensus.cc:493] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:15.023842 2276 raft_consensus.cc:2272] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.024511 2872 raft_consensus.cc:515] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:15.025880 717 leader_election.cc:343] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: ccdde3e5a13e436d9f2328831e155adb
I20260505 07:54:15.026135 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 8b406c4c6ba741bc85eef54c6f576459 (table default.Table_d4d6e315fb8340e9b629d876d050ef05 [id=5fdbc0ad9c3f4e0a9b2fc0997a87870b]) successfully deleted
I20260505 07:54:15.025919 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:15.026935 2872 leader_election.cc:290] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.027532 2276 ts_tablet_manager.cc:1916] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.027467 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
W20260505 07:54:15.030344 718 leader_election.cc:343] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: ccdde3e5a13e436d9f2328831e155adb
I20260505 07:54:15.030483 718 leader_election.cc:304] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:15.030871 2872 raft_consensus.cc:2749] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:15.027837 1172 tablet_bootstrap.cc:492] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
W20260505 07:54:15.032021 856 leader_election.cc:343] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.032249 1354 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 185) took 2412 ms. Trace:
I20260505 07:54:15.032358 1354 rpcz_store.cc:276] 0505 07:54:12.620119 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.880775 (+2260656us) service_pool.cc:224] Handling call
0505 07:54:14.882123 (+ 1348us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:14.882560 (+ 437us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:14.911398 (+ 28838us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:15.032113 (+120715us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:15.032218 (+ 105us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:14.883122 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:14.883222 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:14.883231 (+ 9us) write_op.cc:435] Acquired schema lock
0505 07:54:14.883234 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:14.883345 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:14.883362 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:14.883369 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:14.883466 (+ 97us) tablet.cc:701] Row locks acquired
0505 07:54:14.883469 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:14.883511 (+ 42us) write_op.cc:270] Start()
0505 07:54:14.883553 (+ 42us) write_op.cc:276] Timestamp: P: 1777967654883423 usec, L: 0
0505 07:54:14.883558 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:14.883824 (+ 266us) log.cc:844] Serialized 433 byte log entry
0505 07:54:14.890589 (+ 6765us) op_driver.cc:464] REPLICATION: finished
0505 07:54:14.910791 (+ 20202us) write_op.cc:301] APPLY: starting
0505 07:54:14.910811 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:14.910865 (+ 54us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:14.910924 (+ 59us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:14.911073 (+ 149us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:14.911109 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:14.911116 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:14.911211 (+ 95us) log.cc:844] Serialized 29 byte log entry
0505 07:54:14.911285 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.911308 (+ 23us) write_op.cc:454] Released schema lock
0505 07:54:14.911314 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:14.911397 (+ 83us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.911402 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":20090,"apply.run_cpu_time_us":616,"apply.run_wall_time_us":616,"num_ops":2,"prepare.queue_time_us":67,"prepare.run_cpu_time_us":841,"prepare.run_wall_time_us":841,"raft.queue_time_us":448,"raft.run_cpu_time_us":394,"raft.run_wall_time_us":393,"replication_time_us":7003}]]}
I20260505 07:54:15.033129 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_97a06a334aa7478996655bf7214eb4c1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:15.033783 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_97a06a334aa7478996655bf7214eb4c1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:15.034026 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "139d5a2ab9594f6ea6042c1c4ba8a46a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:15.034227 777 raft_consensus.cc:3060] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:15.035545 856 leader_election.cc:343] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.036973 777 raft_consensus.cc:2468] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:15.037285 854 leader_election.cc:304] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.037635 2853 raft_consensus.cc:2804] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:15.037787 2853 raft_consensus.cc:697] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:15.038008 2853 consensus_queue.cc:237] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.039804 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f845a4a0795e491c937c56e208c7d173. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:15.040832 856 leader_election.cc:343] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.040946 856 leader_election.cc:304] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.041246 2749 raft_consensus.cc:2749] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.042173 2778 raft_consensus.cc:493] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:15.042358 2778 raft_consensus.cc:515] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:15.042889 2778 leader_election.cc:290] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.044238 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 186) took 5488 ms. Trace:
I20260505 07:54:15.044358 2447 rpcz_store.cc:276] 0505 07:54:09.555277 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.558315 (+ 3038us) service_pool.cc:224] Handling call
0505 07:54:14.766360 (+5208045us) ts_tablet_manager.cc:1151] Deleting tablet 43e4c92b7c3140fdb2d6a290aaf0ad55
0505 07:54:14.766369 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.940819 (+174450us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:14.944219 (+ 3400us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.044196 (+ 99977us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5207911}
I20260505 07:54:15.044713 2447 tablet_replica.cc:333] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:15.044942 2447 raft_consensus.cc:2243] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.045085 2447 raft_consensus.cc:2272] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:15.045296 988 leader_election.cc:343] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 23fa401bb4634022bca3c6b23421e4d5
I20260505 07:54:15.045444 988 leader_election.cc:304] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.045605 2447 ts_tablet_manager.cc:1916] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.047842 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 (table default.Table_43b383b7a58b493c8f5b2d9f3038a2e2 [id=5154509860404937ba84ef134bb50cef]) successfully deleted
I20260505 07:54:15.051339 1176 tablet_bootstrap.cc:492] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.051421 2752 raft_consensus.cc:493] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.051529 2752 raft_consensus.cc:515] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.052086 2752 leader_election.cc:290] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.052690 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18e909ee93a24ea9842e95357c2ba858" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.052881 772 raft_consensus.cc:2468] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:15.053103 1176 tablet_bootstrap.cc:654] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.053224 854 leader_election.cc:304] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.053448 2752 raft_consensus.cc:2804] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:15.053505 2752 raft_consensus.cc:493] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:15.053570 2752 raft_consensus.cc:3060] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.054335 2778 raft_consensus.cc:2749] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:15.055847 988 consensus_peers.cc:597] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 397d33e908af4a0bb8dde26d273be01a. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:15.056006 988 leader_election.cc:304] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:15.056159 2752 raft_consensus.cc:515] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.057346 2778 raft_consensus.cc:2804] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:15.057468 2778 raft_consensus.cc:697] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:15.057698 2778 consensus_queue.cc:237] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:15.063462 2505 raft_consensus.cc:493] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.063571 2505 raft_consensus.cc:515] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.064096 2505 leader_election.cc:290] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.064577 2447 ts_tablet_manager.cc:1929] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.064713 2447 log.cc:1199] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/ccdde3e5a13e436d9f2328831e155adb
I20260505 07:54:15.065093 2447 ts_tablet_manager.cc:1950] T ccdde3e5a13e436d9f2328831e155adb P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
W20260505 07:54:15.065759 988 consensus_peers.cc:597] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0db162dfad2448e8ac428b3d3fbe17f7. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:15.065922 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e010f23c4df647ab863e03c3cfed7898" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.066130 772 raft_consensus.cc:2468] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:15.066526 854 leader_election.cc:304] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.066807 2505 raft_consensus.cc:2804] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:15.066876 2505 raft_consensus.cc:493] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:15.066936 2505 raft_consensus.cc:3060] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:15.031715 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:15.069631 2505 raft_consensus.cc:515] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.069562 1354 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 186) took 2200 ms. Trace:
I20260505 07:54:15.070353 2505 leader_election.cc:290] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.069907 1354 rpcz_store.cc:276] 0505 07:54:12.869373 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.032952 (+2163579us) service_pool.cc:224] Handling call
0505 07:54:15.033916 (+ 964us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:15.034336 (+ 420us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:15.065975 (+ 31639us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.065982 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:15.066070 (+ 88us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:15.069412 (+ 3342us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:15.069519 (+ 107us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.034817 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.034945 (+ 128us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.034950 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:15.034953 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:15.035029 (+ 76us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.035041 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.035045 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:15.035111 (+ 66us) tablet.cc:701] Row locks acquired
0505 07:54:15.035114 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:15.035144 (+ 30us) write_op.cc:270] Start()
0505 07:54:15.035175 (+ 31us) write_op.cc:276] Timestamp: P: 1777967655035047 usec, L: 0
0505 07:54:15.035179 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.035327 (+ 148us) log.cc:844] Serialized 433 byte log entry
0505 07:54:15.037885 (+ 2558us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.038013 (+ 128us) write_op.cc:301] APPLY: starting
0505 07:54:15.038028 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.038074 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.038103 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.038218 (+ 115us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.038253 (+ 35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:15.038260 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:15.038330 (+ 70us) log.cc:844] Serialized 29 byte log entry
0505 07:54:15.038387 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.038405 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:15.038411 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":83,"apply.run_cpu_time_us":447,"apply.run_wall_time_us":447,"num_ops":2,"prepare.queue_time_us":60,"prepare.run_cpu_time_us":602,"prepare.run_wall_time_us":603,"raft.queue_time_us":56,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":208,"replication_time_us":2678}]]}
W20260505 07:54:15.070550 856 leader_election.cc:343] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 1602ccbae24544c4a6cf14ea073880b4
I20260505 07:54:15.070652 856 leader_election.cc:304] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:15.070791 856 raft_consensus.cc:2686] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260505 07:54:15.070736 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_040e8b41aef74a32a3a48a5e12a29ba7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:15.071229 856 leader_election.cc:343] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:15.071290 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_040e8b41aef74a32a3a48a5e12a29ba7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:15.071660 856 leader_election.cc:343] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.071849 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e010f23c4df647ab863e03c3cfed7898" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:15.072024 772 raft_consensus.cc:3060] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
W20260505 07:54:15.072086 856 leader_election.cc:343] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 727cacd242844455915bd9963d2db19f
I20260505 07:54:15.072300 2813 catalog_manager.cc:1269] Loaded cluster ID: 9b9d7d3d45334a3e97a4e71f5c281d3c
I20260505 07:54:15.072391 2813 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: loading cluster ID for follower catalog manager: success
W20260505 07:54:15.072435 856 leader_election.cc:343] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 727cacd242844455915bd9963d2db19f
I20260505 07:54:15.074420 772 raft_consensus.cc:2468] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 2.
I20260505 07:54:15.074584 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18e909ee93a24ea9842e95357c2ba858" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.074843 1050 raft_consensus.cc:2468] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:15.074997 854 leader_election.cc:304] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.077200 1181 tablet_bootstrap.cc:492] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.077373 1181 ts_tablet_manager.cc:1403] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.164s user 0.003s sys 0.006s
I20260505 07:54:15.078290 1181 raft_consensus.cc:359] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.078581 1181 raft_consensus.cc:385] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.078744 1181 raft_consensus.cc:740] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.079492 1181 consensus_queue.cc:260] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.083660 2505 raft_consensus.cc:493] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.083835 2505 raft_consensus.cc:515] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.083889 1354 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 187) took 2211 ms. Trace:
I20260505 07:54:15.084002 1354 rpcz_store.cc:276] 0505 07:54:12.872073 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.070558 (+2198485us) service_pool.cc:224] Handling call
0505 07:54:15.071379 (+ 821us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:15.071827 (+ 448us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:15.083567 (+ 11740us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.083573 (+ 6us) write_op.cc:454] Released schema lock
0505 07:54:15.083658 (+ 85us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:15.083765 (+ 107us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:15.083851 (+ 86us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.072445 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.072537 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.072542 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:15.072544 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:15.072624 (+ 80us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.072636 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.072640 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:15.072704 (+ 64us) tablet.cc:701] Row locks acquired
0505 07:54:15.072706 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:15.072735 (+ 29us) write_op.cc:270] Start()
0505 07:54:15.072764 (+ 29us) write_op.cc:276] Timestamp: P: 1777967655072634 usec, L: 0
0505 07:54:15.072767 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.072920 (+ 153us) log.cc:844] Serialized 433 byte log entry
0505 07:54:15.076532 (+ 3612us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.076644 (+ 112us) write_op.cc:301] APPLY: starting
0505 07:54:15.076662 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.076703 (+ 41us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.076725 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.076857 (+ 132us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.076890 (+ 33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:15.076900 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:15.076985 (+ 85us) log.cc:844] Serialized 29 byte log entry
0505 07:54:15.077053 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.077073 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:15.077079 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":58,"apply.run_cpu_time_us":482,"apply.run_wall_time_us":482,"num_ops":2,"prepare.queue_time_us":41,"prepare.run_cpu_time_us":579,"prepare.run_wall_time_us":579,"raft.queue_time_us":983,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":328,"replication_time_us":3735}]]}
I20260505 07:54:15.084637 1354 master_service.cc:954] ReplaceTablet: received request to replace tablet 70051011024f4971bd0166d335d32393 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:15.085182 2505 leader_election.cc:290] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.085366 2505 raft_consensus.cc:2804] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:15.085496 2505 raft_consensus.cc:697] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
W20260505 07:54:15.086248 856 leader_election.cc:343] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: e010f23c4df647ab863e03c3cfed7898
I20260505 07:54:15.087625 1172 ts_tablet_manager.cc:1403] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.126s user 0.005s sys 0.001s
W20260505 07:54:15.087867 856 leader_election.cc:343] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: e010f23c4df647ab863e03c3cfed7898
I20260505 07:54:15.088459 1181 ts_tablet_manager.cc:1434] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.011s user 0.002s sys 0.000s
I20260505 07:54:15.088449 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:15.088618 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:15.065929 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 187) took 5510 ms. Trace:
I20260505 07:54:15.089313 2447 rpcz_store.cc:276] 0505 07:54:09.555348 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.558354 (+ 3006us) service_pool.cc:224] Handling call
0505 07:54:15.044656 (+5486302us) ts_tablet_manager.cc:1151] Deleting tablet ccdde3e5a13e436d9f2328831e155adb
0505 07:54:15.044665 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.061548 (+ 16883us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.064471 (+ 2923us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.065843 (+ 1372us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5486200}
I20260505 07:54:15.089720 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet ccdde3e5a13e436d9f2328831e155adb (table default.Table_efeac090fb094bdf82f547af366d562c [id=047ae44eaf5e48ba8b11e90f6d5f8119]) successfully deleted
I20260505 07:54:15.089896 1181 tablet_bootstrap.cc:492] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.090605 1172 raft_consensus.cc:359] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.090757 1172 raft_consensus.cc:385] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.090806 1172 raft_consensus.cc:740] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:54:15.088634 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 60251.8us
I20260505 07:54:15.088512 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70051011024f4971bd0166d335d32393" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.091015 1172 consensus_queue.cc:260] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:15.091454 856 leader_election.cc:343] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.088896 1357 catalog_manager.cc:5671] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:15.091905 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:15.092207 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:15.092401 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:15.091483 1172 ts_tablet_manager.cc:1434] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20260505 07:54:15.092995 1172 tablet_bootstrap.cc:492] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.093619 2447 tablet_replica.cc:333] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:15.093906 2447 raft_consensus.cc:2243] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.094154 2447 raft_consensus.cc:2272] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.094406 1172 tablet_bootstrap.cc:654] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.094841 2447 ts_tablet_manager.cc:1916] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.095105 1357 catalog_manager.cc:5671] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:15.095393 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:15.095754 1357 catalog_manager.cc:5671] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:15.096086 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:15.096050 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70051011024f4971bd0166d335d32393" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.096343 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
W20260505 07:54:15.096503 854 leader_election.cc:343] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.096616 854 leader_election.cc:304] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.096539 1357 catalog_manager.cc:5671] T a1175f7deec14fe39098ec89ab3bc049 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:15.096807 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:15.096948 2853 raft_consensus.cc:2749] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.096988 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:15.097148 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:15.097616 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:15.097826 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:15.097970 2752 leader_election.cc:290] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.098156 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:15.090380 2268 ts_tablet_manager.cc:1929] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.098351 2268 log.cc:1199] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/bd581a6799a14a8dafb80d411e8936f3
I20260505 07:54:15.098670 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:15.098757 2268 ts_tablet_manager.cc:1950] T bd581a6799a14a8dafb80d411e8936f3 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.098930 1357 catalog_manager.cc:5671] T aa049ad4edbd466a860428c822a52724 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:15.100220 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:15.100379 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:15.100553 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:15.100776 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:15.100611 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18e909ee93a24ea9842e95357c2ba858" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:15.101071 772 raft_consensus.cc:3060] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.101083 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:15.101351 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:15.101750 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:15.102633 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
W20260505 07:54:15.103235 1322 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet b0d966bcd7f1419cbd090afc185ae78a (table default.Table_58eed87aa11e4e3fa7a7d18a135128a0 [id=3ae3bc457dcb4db1b5f4360cb9e5c81c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b0d966bcd7f1419cbd090afc185ae78a consensus state has no leader
I20260505 07:54:15.103914 772 raft_consensus.cc:2468] T 18e909ee93a24ea9842e95357c2ba858 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:15.104465 854 leader_election.cc:304] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.108062 2752 raft_consensus.cc:2804] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:15.108148 2276 ts_tablet_manager.cc:1929] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.108170 2752 raft_consensus.cc:697] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:15.108223 2276 log.cc:1199] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/1602ccbae24544c4a6cf14ea073880b4
I20260505 07:54:15.108453 1176 tablet_bootstrap.cc:492] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.108387 2752 consensus_queue.cc:237] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.108527 2276 ts_tablet_manager.cc:1950] T 1602ccbae24544c4a6cf14ea073880b4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.108621 1176 ts_tablet_manager.cc:1403] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.057s user 0.003s sys 0.003s
I20260505 07:54:15.109266 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 116) took 7327 ms. Trace:
I20260505 07:54:15.109338 2276 rpcz_store.cc:276] 0505 07:54:07.782055 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.800015 (+ 17960us) service_pool.cc:224] Handling call
0505 07:54:15.021465 (+7221450us) ts_tablet_manager.cc:1151] Deleting tablet 1602ccbae24544c4a6cf14ea073880b4
0505 07:54:15.021642 (+ 177us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.103649 (+ 82007us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.108103 (+ 4454us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.109250 (+ 1147us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7221313}
I20260505 07:54:15.109372 1176 raft_consensus.cc:359] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.109531 1176 raft_consensus.cc:385] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.109640 1176 raft_consensus.cc:740] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.109659 2276 tablet_replica.cc:333] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.109836 1176 consensus_queue.cc:260] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.110313 1176 ts_tablet_manager.cc:1434] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:15.111764 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 1602ccbae24544c4a6cf14ea073880b4 (table default.Table_e9ba8a72d90d4a3da2c06b684fe35033 [id=5c8387e5e7f84731afc62bbea6475a85]) successfully deleted
W20260505 07:54:15.112812 991 leader_election.cc:343] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: cac5064f68d24ebe95b5a0b8e10f463e
I20260505 07:54:15.113094 2505 consensus_queue.cc:237] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.114277 2872 raft_consensus.cc:493] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.114415 2872 raft_consensus.cc:515] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.114933 2872 leader_election.cc:290] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.115312 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:15.117481 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:15.118191 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:15.118487 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac5064f68d24ebe95b5a0b8e10f463e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.118570 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:15.118695 777 raft_consensus.cc:2468] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:15.118824 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:15.121322 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:15.121865 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:15.123147 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:15.123394 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:15.123624 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:15.124034 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:15.124496 1176 tablet_bootstrap.cc:492] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.124889 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
W20260505 07:54:15.125370 718 leader_election.cc:343] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 27bacc97d7534284970f218cf68ac75b
I20260505 07:54:15.125438 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:15.125614 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:15.125936 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:15.126186 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:15.126457 1357 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:15.136402 1181 tablet_bootstrap.cc:654] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:15.143713 718 consensus_peers.cc:597] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f92c53c68a4b4e319e98b84128e8e6a4. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:15.147653 1176 tablet_bootstrap.cc:654] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.147925 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bacc97d7534284970f218cf68ac75b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:15.148540 717 leader_election.cc:343] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.148670 717 leader_election.cc:304] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:15.149137 2674 raft_consensus.cc:2749] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.151664 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 150) took 5334 ms. Trace:
I20260505 07:54:15.151796 2268 rpcz_store.cc:276] 0505 07:54:09.816957 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.817282 (+ 325us) service_pool.cc:224] Handling call
0505 07:54:14.955586 (+5138304us) ts_tablet_manager.cc:1151] Deleting tablet bd581a6799a14a8dafb80d411e8936f3
0505 07:54:14.955623 (+ 37us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:14.967347 (+ 11724us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.090314 (+122967us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.151629 (+ 61315us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5138173}
I20260505 07:54:15.152163 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet bd581a6799a14a8dafb80d411e8936f3 (table default.Table_822258eaa3b14528a8bfa97e3e9d4d7a [id=76b0eb99a1f343829e5ba35de6302cf2]) successfully deleted
I20260505 07:54:15.153142 988 leader_election.cc:304] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:15.153462 2778 raft_consensus.cc:2804] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:15.153559 2778 raft_consensus.cc:493] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
W20260505 07:54:15.156522 991 consensus_peers.cc:597] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf76e868bbcb44b0b1887c84dd27f093. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:15.153684 1176 tablet_bootstrap.cc:492] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.167820 1176 ts_tablet_manager.cc:1403] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.043s user 0.002s sys 0.002s
I20260505 07:54:15.168807 1176 raft_consensus.cc:359] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.169001 1176 raft_consensus.cc:385] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.169064 1176 raft_consensus.cc:740] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.169294 1176 consensus_queue.cc:260] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.169879 1176 ts_tablet_manager.cc:1434] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:15.170157 1176 tablet_bootstrap.cc:492] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.170665 1181 tablet_bootstrap.cc:492] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.171846 2778 raft_consensus.cc:3060] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:15.175742 1354 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 70051011024f4971bd0166d335d32393
I20260505 07:54:15.176254 1354 catalog_manager.cc:6741] ReplaceTablet: tablet 70051011024f4971bd0166d335d32393 deleted and replaced by tablet 9c37e8de609141c2b871694b2873a746
I20260505 07:54:15.176383 1354 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 188) took 2292 ms. Trace:
I20260505 07:54:15.176785 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 70051011024f4971bd0166d335d32393 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c37e8de609141c2b871694b2873a746) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:15.176465 1354 rpcz_store.cc:276] 0505 07:54:12.883739 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.084569 (+2200830us) service_pool.cc:224] Handling call
0505 07:54:15.176331 (+ 91762us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.167284 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.167434 (+ 150us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.167442 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:15.167445 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:15.167607 (+ 162us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.167623 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.167627 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:15.167704 (+ 77us) tablet.cc:701] Row locks acquired
0505 07:54:15.167706 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:15.167755 (+ 49us) write_op.cc:270] Start()
0505 07:54:15.167812 (+ 57us) write_op.cc:276] Timestamp: P: 1777967655167645 usec, L: 0
0505 07:54:15.167816 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.168018 (+ 202us) log.cc:844] Serialized 622 byte log entry
0505 07:54:15.174177 (+ 6159us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.174291 (+ 114us) write_op.cc:301] APPLY: starting
0505 07:54:15.174305 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.174349 (+ 44us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.174371 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.174474 (+ 103us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.174544 (+ 70us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:15.174551 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:15.174622 (+ 71us) log.cc:844] Serialized 29 byte log entry
0505 07:54:15.174679 (+ 57us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.174699 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:15.174706 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:15.174791 (+ 85us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.174795 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":70,"apply.run_cpu_time_us":491,"apply.run_wall_time_us":491,"num_ops":2,"prepare.queue_time_us":61,"prepare.run_cpu_time_us":856,"prepare.run_wall_time_us":856,"raft.queue_time_us":1322,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":460,"replication_time_us":6333}]]}
I20260505 07:54:15.177350 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 70051011024f4971bd0166d335d32393 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c37e8de609141c2b871694b2873a746) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:15.177516 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_48ab1b77fb0544cf81d96c1b8bd9a997"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:15.177879 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 70051011024f4971bd0166d335d32393 with delete_type TABLET_DATA_DELETED (replaced by tablet 9c37e8de609141c2b871694b2873a746) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:15.178043 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48ab1b77fb0544cf81d96c1b8bd9a997 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:15.178352 2778 raft_consensus.cc:515] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:15.178911 2778 leader_election.cc:290] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:15.179651 991 leader_election.cc:343] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: cac5064f68d24ebe95b5a0b8e10f463e
I20260505 07:54:15.173358 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18e909ee93a24ea9842e95357c2ba858" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:15.180001 1048 raft_consensus.cc:3060] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.182415 1048 raft_consensus.cc:2468] T 18e909ee93a24ea9842e95357c2ba858 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:15.182518 2856 raft_consensus.cc:493] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.182687 2856 raft_consensus.cc:515] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.183719 2856 leader_election.cc:290] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.183809 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1506cd31fe364e328867bbf70e36845e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:15.184495 854 leader_election.cc:343] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.186105 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cac5064f68d24ebe95b5a0b8e10f463e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:15.186331 772 raft_consensus.cc:3060] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:15.189225 2975 tablet.cc:1724] T 3a73dbb979344805876efbafbf1b2985 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:15.189884 2670 consensus_queue.cc:1048] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.189940 2976 tablet.cc:1724] T 3a73dbb979344805876efbafbf1b2985 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:15.193496 1354 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 189) took 2302 ms. Trace:
I20260505 07:54:15.193621 1354 rpcz_store.cc:276] 0505 07:54:12.891199 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.177047 (+2285848us) service_pool.cc:224] Handling call
0505 07:54:15.178139 (+ 1092us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:15.178577 (+ 438us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:15.193238 (+ 14661us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:15.193353 (+ 115us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:15.193439 (+ 86us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.181512 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.181605 (+ 93us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.181609 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:15.181611 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:15.181683 (+ 72us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.181693 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.181697 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:15.181757 (+ 60us) tablet.cc:701] Row locks acquired
0505 07:54:15.181760 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:15.181789 (+ 29us) write_op.cc:270] Start()
0505 07:54:15.181821 (+ 32us) write_op.cc:276] Timestamp: P: 1777967655181674 usec, L: 0
0505 07:54:15.181826 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.182014 (+ 188us) log.cc:844] Serialized 433 byte log entry
0505 07:54:15.188627 (+ 6613us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.188750 (+ 123us) write_op.cc:301] APPLY: starting
0505 07:54:15.188763 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.188799 (+ 36us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.188830 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.188918 (+ 88us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.188967 (+ 49us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:15.188975 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:15.189037 (+ 62us) log.cc:844] Serialized 29 byte log entry
0505 07:54:15.189090 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.189108 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:15.189115 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:15.189173 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.189177 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":65,"apply.run_cpu_time_us":415,"apply.run_wall_time_us":415,"num_ops":2,"prepare.queue_time_us":2483,"prepare.run_cpu_time_us":598,"prepare.run_wall_time_us":597,"raft.queue_time_us":392,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":286,"replication_time_us":6777}]]}
I20260505 07:54:15.194532 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_1d71a1e96bb3432b9441aa916af3a79c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:15.195246 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1d71a1e96bb3432b9441aa916af3a79c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:15.200841 1176 tablet_bootstrap.cc:654] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.171583 1181 ts_tablet_manager.cc:1403] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.082s user 0.006s sys 0.000s
I20260505 07:54:15.185321 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1506cd31fe364e328867bbf70e36845e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.202543 1181 raft_consensus.cc:359] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.202705 1181 raft_consensus.cc:385] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.202777 1181 raft_consensus.cc:740] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.204742 1354 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 190) took 2310 ms. Trace:
I20260505 07:54:15.188730 2974 tablet.cc:1724] T 3a73dbb979344805876efbafbf1b2985 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:15.204845 1354 rpcz_store.cc:276] 0505 07:54:12.894653 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.194349 (+2299696us) service_pool.cc:224] Handling call
0505 07:54:15.195365 (+ 1016us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:15.195830 (+ 465us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:15.204397 (+ 8567us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.204404 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:15.204467 (+ 63us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:15.204599 (+ 132us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:15.204677 (+ 78us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.196329 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.196419 (+ 90us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.196424 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:15.196426 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:15.196498 (+ 72us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.196509 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.196513 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:15.196577 (+ 64us) tablet.cc:701] Row locks acquired
0505 07:54:15.196579 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:15.196604 (+ 25us) write_op.cc:270] Start()
0505 07:54:15.196628 (+ 24us) write_op.cc:276] Timestamp: P: 1777967655196492 usec, L: 0
0505 07:54:15.196631 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.196763 (+ 132us) log.cc:844] Serialized 433 byte log entry
0505 07:54:15.200377 (+ 3614us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.200483 (+ 106us) write_op.cc:301] APPLY: starting
0505 07:54:15.200496 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.200542 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.200564 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.200653 (+ 89us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.200680 (+ 27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:15.200687 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:15.200750 (+ 63us) log.cc:844] Serialized 29 byte log entry
0505 07:54:15.200802 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.200819 (+ 17us) write_op.cc:454] Released schema lock
0505 07:54:15.200826 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":64,"apply.run_cpu_time_us":390,"apply.run_wall_time_us":389,"num_ops":2,"prepare.queue_time_us":41,"prepare.run_cpu_time_us":521,"prepare.run_wall_time_us":522,"raft.queue_time_us":385,"raft.run_cpu_time_us":420,"raft.run_wall_time_us":419,"replication_time_us":3723}]]}
I20260505 07:54:15.205598 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_da1e2e5149fb4ffba207cdfa95973244"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:15.195436 2268 tablet_replica.cc:333] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.197726 2742 raft_consensus.cc:493] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.205997 2742 raft_consensus.cc:515] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:15.206111 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da1e2e5149fb4ffba207cdfa95973244 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:15.206728 2742 leader_election.cc:290] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.207298 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3462dcc6391741e59dcb5cd08527e50f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.207499 920 raft_consensus.cc:2393] T 3462dcc6391741e59dcb5cd08527e50f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:15.208182 2268 raft_consensus.cc:2243] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.208433 2268 raft_consensus.cc:2272] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.209084 2268 ts_tablet_manager.cc:1916] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:15.211741 856 leader_election.cc:343] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.211846 856 leader_election.cc:304] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.212052 2856 raft_consensus.cc:2749] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.203473 1181 consensus_queue.cc:260] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.213116 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3462dcc6391741e59dcb5cd08527e50f" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.213474 1181 ts_tablet_manager.cc:1434] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260505 07:54:15.214361 988 leader_election.cc:304] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.215969 2778 raft_consensus.cc:3060] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.216840 2447 ts_tablet_manager.cc:1929] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.216925 2447 log.cc:1199] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/36b33d265ba04e93993c7e0a4efea1a4
I20260505 07:54:15.217226 2447 ts_tablet_manager.cc:1950] T 36b33d265ba04e93993c7e0a4efea1a4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:15.217259 1354 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 191) took 2293 ms. Trace:
I20260505 07:54:15.217347 1354 rpcz_store.cc:276] 0505 07:54:12.923762 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.205427 (+2281665us) service_pool.cc:224] Handling call
0505 07:54:15.206208 (+ 781us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:15.206596 (+ 388us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:15.216878 (+ 10282us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.216883 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:15.216956 (+ 73us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:15.217125 (+ 169us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:15.217205 (+ 80us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.207120 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.207220 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.207224 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:15.207226 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:15.207318 (+ 92us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.207330 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.207335 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:15.207395 (+ 60us) tablet.cc:701] Row locks acquired
0505 07:54:15.207398 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:15.207427 (+ 29us) write_op.cc:270] Start()
0505 07:54:15.207454 (+ 27us) write_op.cc:276] Timestamp: P: 1777967655207313 usec, L: 0
0505 07:54:15.207458 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.207650 (+ 192us) log.cc:844] Serialized 433 byte log entry
0505 07:54:15.211396 (+ 3746us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.211489 (+ 93us) write_op.cc:301] APPLY: starting
0505 07:54:15.211501 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.211536 (+ 35us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.211566 (+ 30us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.211685 (+ 119us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.211713 (+ 28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:15.211719 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:15.211779 (+ 60us) log.cc:844] Serialized 29 byte log entry
0505 07:54:15.211839 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.211860 (+ 21us) write_op.cc:454] Released schema lock
0505 07:54:15.211867 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":55,"apply.run_cpu_time_us":430,"apply.run_wall_time_us":429,"num_ops":2,"prepare.queue_time_us":41,"prepare.run_cpu_time_us":622,"prepare.run_wall_time_us":623,"raft.queue_time_us":370,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":323,"replication_time_us":3915}]]}
I20260505 07:54:15.217989 1176 tablet_bootstrap.cc:492] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.218094 1176 ts_tablet_manager.cc:1403] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.048s user 0.000s sys 0.002s
I20260505 07:54:15.218111 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_4c7d78e1cc634a52ba8de34b2f90b809"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:15.218442 2778 raft_consensus.cc:2749] T 3462dcc6391741e59dcb5cd08527e50f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:15.218616 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c7d78e1cc634a52ba8de34b2f90b809 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:15.218834 1176 raft_consensus.cc:359] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.219039 1176 raft_consensus.cc:385] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.219089 1176 raft_consensus.cc:740] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.219297 1176 consensus_queue.cc:260] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.219373 2276 raft_consensus.cc:2243] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.219746 1176 ts_tablet_manager.cc:1434] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:15.219786 2276 pending_rounds.cc:70] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:54:15.219846 2276 pending_rounds.cc:77] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 2 index: 1 } timestamp: 7282555515345645568 op_type: NO_OP noop_request { }
I20260505 07:54:15.219957 2276 raft_consensus.cc:2889] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:15.220008 1176 tablet_bootstrap.cc:492] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.220037 2276 raft_consensus.cc:2272] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.221298 1176 tablet_bootstrap.cc:654] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.226559 2276 ts_tablet_manager.cc:1916] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.227449 772 raft_consensus.cc:2468] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 2.
I20260505 07:54:15.230187 1181 tablet_bootstrap.cc:492] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.231833 1181 tablet_bootstrap.cc:654] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.231971 2276 ts_tablet_manager.cc:1929] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260505 07:54:15.232066 2276 log.cc:1199] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e010f23c4df647ab863e03c3cfed7898
I20260505 07:54:15.232381 2276 ts_tablet_manager.cc:1950] T e010f23c4df647ab863e03c3cfed7898 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.232641 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 117) took 7450 ms. Trace:
I20260505 07:54:15.232714 2276 rpcz_store.cc:276] 0505 07:54:07.782115 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.800173 (+ 18058us) service_pool.cc:224] Handling call
0505 07:54:15.109619 (+7309446us) ts_tablet_manager.cc:1151] Deleting tablet e010f23c4df647ab863e03c3cfed7898
0505 07:54:15.109629 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.219608 (+109979us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x616000107854. stack: 00007f1644039086 00007f1644038e40 00007f1642ff23ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007f164ff7c7e0 00007f16512d0283 00007f1653e95a77 00007f1653ec8b44 00007f1653ea9560 00007f1653ea92ed 0000000000aa4cac 00007f164407ef6f 00007f164408228b 00007f1644082108
0505 07:54:15.229426 (+ 9818us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.231924 (+ 2498us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.232628 (+ 704us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9498,"spinlock_wait_cycles":230014720,"tablet-delete.queue_time_us":7309319}
I20260505 07:54:15.233148 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 118) took 7450 ms. Trace:
I20260505 07:54:15.233173 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e010f23c4df647ab863e03c3cfed7898 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]) successfully deleted
I20260505 07:54:15.233217 2276 rpcz_store.cc:276] 0505 07:54:07.782172 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.800310 (+ 18138us) service_pool.cc:224] Handling call
0505 07:54:15.233041 (+7432731us) ts_tablet_manager.cc:1151] Deleting tablet f2f5b72f58a24db0b58c15b47227bc60
0505 07:54:15.233050 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.233134 (+ 84us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7432579}
I20260505 07:54:15.233434 2276 tablet_replica.cc:333] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.233596 2276 raft_consensus.cc:2243] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.233722 2276 pending_rounds.cc:70] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:54:15.233755 2276 pending_rounds.cc:77] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555497389105152 op_type: NO_OP noop_request { }
I20260505 07:54:15.233820 2276 raft_consensus.cc:2889] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260505 07:54:15.233788 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet f2f5b72f58a24db0b58c15b47227bc60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2f5b72f58a24db0b58c15b47227bc60 already in progress: creating tablet
I20260505 07:54:15.233896 2276 raft_consensus.cc:2272] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.234354 2276 ts_tablet_manager.cc:1916] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.234658 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 189) took 5679 ms. Trace:
I20260505 07:54:15.234750 2447 rpcz_store.cc:276] 0505 07:54:09.555495 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.558461 (+ 2966us) service_pool.cc:224] Handling call
0505 07:54:15.091675 (+5533214us) ts_tablet_manager.cc:1151] Deleting tablet 36b33d265ba04e93993c7e0a4efea1a4
0505 07:54:15.093544 (+ 1869us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.103414 (+ 9870us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.216790 (+113376us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.234639 (+ 17849us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5533113}
I20260505 07:54:15.234862 988 leader_election.cc:304] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:15.235162 2447 tablet_replica.cc:333] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:15.235114 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 36b33d265ba04e93993c7e0a4efea1a4 (table default.Table_0b3a73b640a644579fa864cce1a7ecbe [id=6c1e059a94014956ab0b1d6c750d8494]) successfully deleted
I20260505 07:54:15.235337 2405 raft_consensus.cc:2804] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:15.235340 2447 raft_consensus.cc:2243] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.235452 2405 raft_consensus.cc:697] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:15.235468 2447 raft_consensus.cc:2272] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.235697 2405 consensus_queue.cc:237] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:15.235921 2447 ts_tablet_manager.cc:1916] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.248533 1172 tablet_bootstrap.cc:492] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.248673 1172 ts_tablet_manager.cc:1403] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.156s user 0.004s sys 0.011s
I20260505 07:54:15.249468 1172 raft_consensus.cc:359] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.249624 1172 raft_consensus.cc:385] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.249675 1172 raft_consensus.cc:740] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.249861 1172 consensus_queue.cc:260] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.250327 1172 ts_tablet_manager.cc:1434] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:15.251713 1176 tablet_bootstrap.cc:492] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.251863 1176 ts_tablet_manager.cc:1403] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.000s
I20260505 07:54:15.252486 1176 raft_consensus.cc:359] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.252676 1176 raft_consensus.cc:385] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.252735 1176 raft_consensus.cc:740] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.252926 1176 consensus_queue.cc:260] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.253439 1176 ts_tablet_manager.cc:1434] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260505 07:54:15.253708 1176 tablet_bootstrap.cc:492] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.253990 1172 tablet_bootstrap.cc:492] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.254967 1176 tablet_bootstrap.cc:654] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.255308 1172 tablet_bootstrap.cc:654] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.258394 2670 consensus_queue.cc:1048] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.266121 2813 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: acquiring CA information for follower catalog manager: success
I20260505 07:54:15.268415 1354 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 192) took 2337 ms. Trace:
I20260505 07:54:15.268534 1354 rpcz_store.cc:276] 0505 07:54:12.931121 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.217921 (+2286800us) service_pool.cc:224] Handling call
0505 07:54:15.218716 (+ 795us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:15.219189 (+ 473us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:15.268138 (+ 48949us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:15.268247 (+ 109us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:15.268344 (+ 97us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.250961 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.251103 (+ 142us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.251109 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:15.251112 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:15.251204 (+ 92us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.251217 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.251221 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:15.251303 (+ 82us) tablet.cc:701] Row locks acquired
0505 07:54:15.251305 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:15.251355 (+ 50us) write_op.cc:270] Start()
0505 07:54:15.251388 (+ 33us) write_op.cc:276] Timestamp: P: 1777967655251234 usec, L: 0
0505 07:54:15.251391 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.251568 (+ 177us) log.cc:844] Serialized 433 byte log entry
0505 07:54:15.258731 (+ 7163us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.258851 (+ 120us) write_op.cc:301] APPLY: starting
0505 07:54:15.258869 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.258916 (+ 47us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.258965 (+ 49us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.259088 (+ 123us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.259128 (+ 40us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:15.259140 (+ 12us) write_op.cc:312] APPLY: finished
0505 07:54:15.259216 (+ 76us) log.cc:844] Serialized 29 byte log entry
0505 07:54:15.259276 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.259296 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:15.259303 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:15.259372 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.259378 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":69,"apply.run_cpu_time_us":512,"apply.run_wall_time_us":512,"num_ops":2,"prepare.queue_time_us":53,"prepare.run_cpu_time_us":720,"prepare.run_wall_time_us":720,"raft.queue_time_us":1173,"raft.run_cpu_time_us":460,"raft.run_wall_time_us":459,"replication_time_us":7307}]]}
I20260505 07:54:15.269410 1354 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_e13508677ec64468adda12ac9efaeb05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:15.270004 1354 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e13508677ec64468adda12ac9efaeb05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:15.271495 2276 ts_tablet_manager.cc:1929] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.271623 2276 log.cc:1199] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/5ad8a79245ad47738d079752e51aed2b
I20260505 07:54:15.271978 2276 ts_tablet_manager.cc:1950] T 5ad8a79245ad47738d079752e51aed2b P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.272675 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 119) took 7490 ms. Trace:
I20260505 07:54:15.272753 2276 rpcz_store.cc:276] 0505 07:54:07.782242 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.800438 (+ 18196us) service_pool.cc:224] Handling call
0505 07:54:15.233400 (+7432962us) ts_tablet_manager.cc:1151] Deleting tablet 5ad8a79245ad47738d079752e51aed2b
0505 07:54:15.233407 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.237267 (+ 3860us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.271433 (+ 34166us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.272658 (+ 1225us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7432857}
I20260505 07:54:15.273047 2276 tablet_replica.cc:333] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.273186 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 5ad8a79245ad47738d079752e51aed2b (table default.Table_ba3de8258d7e46c1ad3a509b8e00b806 [id=75d1988e3a9e4ff3a8c206a662f55131]) successfully deleted
I20260505 07:54:15.273241 2276 raft_consensus.cc:2243] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.274715 1181 tablet_bootstrap.cc:492] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.274844 1181 ts_tablet_manager.cc:1403] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.045s user 0.003s sys 0.002s
I20260505 07:54:15.275543 1181 raft_consensus.cc:359] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.275713 1181 raft_consensus.cc:385] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.275779 1181 raft_consensus.cc:740] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.276558 1181 consensus_queue.cc:260] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.277156 1181 ts_tablet_manager.cc:1434] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:15.284361 2268 ts_tablet_manager.cc:1929] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.286442 1176 tablet_bootstrap.cc:492] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.286578 1176 ts_tablet_manager.cc:1403] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.001s
I20260505 07:54:15.287189 1176 raft_consensus.cc:359] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.287340 1176 raft_consensus.cc:385] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.287385 1176 raft_consensus.cc:740] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.287547 1176 consensus_queue.cc:260] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.288045 1176 ts_tablet_manager.cc:1434] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260505 07:54:15.288287 1176 tablet_bootstrap.cc:492] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.289691 1176 tablet_bootstrap.cc:654] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.289981 1354 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 193) took 2309 ms. Trace:
I20260505 07:54:15.290076 1354 rpcz_store.cc:276] 0505 07:54:12.980926 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.269195 (+2288269us) service_pool.cc:224] Handling call
0505 07:54:15.270533 (+ 1338us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:15.271065 (+ 532us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:15.289618 (+ 18553us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.289625 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:15.289696 (+ 71us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:15.289805 (+ 109us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:15.289896 (+ 91us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.278106 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.278204 (+ 98us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.278213 (+ 9us) write_op.cc:435] Acquired schema lock
0505 07:54:15.278217 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:15.278330 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.278346 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.278353 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:15.278449 (+ 96us) tablet.cc:701] Row locks acquired
0505 07:54:15.278453 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:15.278489 (+ 36us) write_op.cc:270] Start()
0505 07:54:15.278519 (+ 30us) write_op.cc:276] Timestamp: P: 1777967655278366 usec, L: 0
0505 07:54:15.278522 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.278741 (+ 219us) log.cc:844] Serialized 433 byte log entry
0505 07:54:15.284585 (+ 5844us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.284685 (+ 100us) write_op.cc:301] APPLY: starting
0505 07:54:15.284700 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.284742 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.284765 (+ 23us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.284854 (+ 89us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.284886 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:15.284894 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:15.284975 (+ 81us) log.cc:844] Serialized 29 byte log entry
0505 07:54:15.285036 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.285054 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:15.285062 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":55,"apply.run_cpu_time_us":427,"apply.run_wall_time_us":426,"num_ops":2,"prepare.queue_time_us":40,"prepare.run_cpu_time_us":753,"prepare.run_wall_time_us":753,"raft.queue_time_us":1202,"raft.run_cpu_time_us":576,"raft.run_wall_time_us":576,"replication_time_us":6026}]]}
I20260505 07:54:15.290776 1354 master_service.cc:954] ReplaceTablet: received request to replace tablet 72da22d790624b07bfca6f6b6301c925 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:15.291082 2268 log.cc:1199] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/0459f2d45a2d44f1a73e0a0fff53866c
I20260505 07:54:15.291683 2268 ts_tablet_manager.cc:1950] T 0459f2d45a2d44f1a73e0a0fff53866c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.300683 2276 raft_consensus.cc:2272] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.301226 2276 ts_tablet_manager.cc:1916] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.301359 1181 tablet_bootstrap.cc:492] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.306768 1181 tablet_bootstrap.cc:654] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.309624 1172 tablet_bootstrap.cc:492] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.309758 1172 ts_tablet_manager.cc:1403] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.056s user 0.003s sys 0.000s
I20260505 07:54:15.310429 1172 raft_consensus.cc:359] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.310583 1172 raft_consensus.cc:385] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.310650 1172 raft_consensus.cc:740] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.310858 1172 consensus_queue.cc:260] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.314608 1172 ts_tablet_manager.cc:1434] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:15.319959 1181 tablet_bootstrap.cc:492] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.320708 1181 ts_tablet_manager.cc:1403] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.001s
I20260505 07:54:15.324293 1176 tablet_bootstrap.cc:492] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.310503 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 151) took 5492 ms. Trace:
I20260505 07:54:15.324435 1176 ts_tablet_manager.cc:1403] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.001s
I20260505 07:54:15.324470 2268 rpcz_store.cc:276] 0505 07:54:09.817569 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.818637 (+ 1068us) service_pool.cc:224] Handling call
0505 07:54:15.152053 (+5333416us) ts_tablet_manager.cc:1151] Deleting tablet 0459f2d45a2d44f1a73e0a0fff53866c
0505 07:54:15.156347 (+ 4294us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.248784 (+ 92437us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.284288 (+ 35504us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.310468 (+ 26180us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5333277}
I20260505 07:54:15.324905 1172 tablet_bootstrap.cc:492] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.325217 1176 raft_consensus.cc:359] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.325380 1176 raft_consensus.cc:385] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.325433 1176 raft_consensus.cc:740] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.325551 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 0459f2d45a2d44f1a73e0a0fff53866c (table default.Table_01a5eabcd2ff41fca0d5c94e26775754 [id=0a65eba2d4074216964f45dbb0217d7a]) successfully deleted
I20260505 07:54:15.325711 1176 consensus_queue.cc:260] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.326152 1176 ts_tablet_manager.cc:1434] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:15.326761 1172 tablet_bootstrap.cc:654] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.329260 2268 tablet_replica.cc:333] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.329591 2268 raft_consensus.cc:2243] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.329670 2447 ts_tablet_manager.cc:1929] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.329782 2447 log.cc:1199] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/202a237ca580482c98803683a77d96d2
I20260505 07:54:15.329923 2268 raft_consensus.cc:2272] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.330138 2447 ts_tablet_manager.cc:1950] T 202a237ca580482c98803683a77d96d2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:15.330559 2268 ts_tablet_manager.cc:1916] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.333767 2276 ts_tablet_manager.cc:1929] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.333858 2276 log.cc:1199] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/d622612bdc404727a27b83769bf39738
I20260505 07:54:15.334170 2276 ts_tablet_manager.cc:1950] T d622612bdc404727a27b83769bf39738 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.335071 1176 tablet_bootstrap.cc:492] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.336611 1176 tablet_bootstrap.cc:654] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.337364 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 190) took 5781 ms. Trace:
I20260505 07:54:15.325914 1181 raft_consensus.cc:359] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.337457 2447 rpcz_store.cc:276] 0505 07:54:09.555569 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.558566 (+ 2997us) service_pool.cc:224] Handling call
0505 07:54:15.235118 (+5676552us) ts_tablet_manager.cc:1151] Deleting tablet 202a237ca580482c98803683a77d96d2
0505 07:54:15.235128 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.280592 (+ 45464us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.329616 (+ 49024us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.337347 (+ 7731us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5676407}
I20260505 07:54:15.337605 1181 raft_consensus.cc:385] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.337805 1181 raft_consensus.cc:740] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.337900 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 202a237ca580482c98803683a77d96d2 (table default.Table_2f6f14275b8847be96f68a761156a854 [id=9c830b1f855348299c40b40610589233]) successfully deleted
I20260505 07:54:15.338080 1181 consensus_queue.cc:260] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.338579 1181 ts_tablet_manager.cc:1434] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.015s user 0.001s sys 0.000s
I20260505 07:54:15.343657 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 120) took 7560 ms. Trace:
I20260505 07:54:15.343767 2276 rpcz_store.cc:276] 0505 07:54:07.782862 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.800560 (+ 17698us) service_pool.cc:224] Handling call
0505 07:54:15.273007 (+7472447us) ts_tablet_manager.cc:1151] Deleting tablet d622612bdc404727a27b83769bf39738
0505 07:54:15.273016 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.320066 (+ 47050us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.333722 (+ 13656us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.343630 (+ 9908us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7472310}
I20260505 07:54:15.344069 2276 tablet_replica.cc:333] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.344197 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet d622612bdc404727a27b83769bf39738 (table default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]) successfully deleted
I20260505 07:54:15.344250 2276 raft_consensus.cc:2243] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.344551 1181 tablet_bootstrap.cc:492] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.344566 2276 raft_consensus.cc:2272] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.346177 1181 tablet_bootstrap.cc:654] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.349162 2447 tablet_replica.cc:333] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:15.349361 2447 raft_consensus.cc:2243] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.349499 2447 raft_consensus.cc:2272] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.349965 2447 ts_tablet_manager.cc:1916] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.352113 1172 tablet_bootstrap.cc:492] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.352228 1172 ts_tablet_manager.cc:1403] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.002s
I20260505 07:54:15.352880 1172 raft_consensus.cc:359] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.353039 1172 raft_consensus.cc:385] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.353093 1172 raft_consensus.cc:740] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.353283 1172 consensus_queue.cc:260] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.356004 1172 ts_tablet_manager.cc:1434] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:15.356307 1172 tablet_bootstrap.cc:492] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.358421 2813 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 28d3eda04f9c47ae976d19c218b6dcc5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260505 07:54:15.363317 1172 tablet_bootstrap.cc:654] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.365388 1176 tablet_bootstrap.cc:492] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.365520 1176 ts_tablet_manager.cc:1403] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20260505 07:54:15.366083 2674 consensus_queue.cc:1048] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.366209 1176 raft_consensus.cc:359] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.366351 1176 raft_consensus.cc:385] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.366397 1176 raft_consensus.cc:740] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.366546 1176 consensus_queue.cc:260] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.366984 1176 ts_tablet_manager.cc:1434] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:15.367209 1176 tablet_bootstrap.cc:492] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.368645 1176 tablet_bootstrap.cc:654] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.372334 2831 consensus_queue.cc:1048] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.375854 2276 ts_tablet_manager.cc:1916] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.376439 2268 ts_tablet_manager.cc:1929] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.379706 2268 log.cc:1199] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/93d878ae0ddb42319560f74e0a490321
I20260505 07:54:15.381513 2268 ts_tablet_manager.cc:1950] T 93d878ae0ddb42319560f74e0a490321 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.382900 2742 consensus_queue.cc:1048] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.384070 1172 tablet_bootstrap.cc:492] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.384183 1172 ts_tablet_manager.cc:1403] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.001s
I20260505 07:54:15.384658 2856 raft_consensus.cc:493] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.384770 2856 raft_consensus.cc:515] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.385345 2856 leader_election.cc:290] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.385787 1172 raft_consensus.cc:359] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.385914 1172 raft_consensus.cc:385] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.385890 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e08cf03ea0b495a94c6566ea6c8caad" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.386000 1172 raft_consensus.cc:740] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.386168 1172 consensus_queue.cc:260] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:15.386339 856 leader_election.cc:343] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.386713 1172 ts_tablet_manager.cc:1434] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:15.386961 1172 tablet_bootstrap.cc:492] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.388767 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e08cf03ea0b495a94c6566ea6c8caad" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.383566 1181 tablet_bootstrap.cc:492] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.389097 1172 tablet_bootstrap.cc:654] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.389120 1181 ts_tablet_manager.cc:1403] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.045s user 0.002s sys 0.001s
W20260505 07:54:15.389478 854 leader_election.cc:343] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.389595 854 leader_election.cc:304] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.390290 1181 raft_consensus.cc:359] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.391065 1181 raft_consensus.cc:385] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.391268 1181 raft_consensus.cc:740] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.391660 1181 consensus_queue.cc:260] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.391783 2856 raft_consensus.cc:2749] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.392961 1181 ts_tablet_manager.cc:1434] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260505 07:54:15.393616 1181 tablet_bootstrap.cc:492] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.393980 2674 consensus_queue.cc:1048] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.397302 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 152) took 5578 ms. Trace:
I20260505 07:54:15.403054 1172 tablet_bootstrap.cc:492] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.403178 1172 ts_tablet_manager.cc:1403] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.001s
I20260505 07:54:15.403838 1172 raft_consensus.cc:359] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.403990 1172 raft_consensus.cc:385] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.404042 1172 raft_consensus.cc:740] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.404207 1172 consensus_queue.cc:260] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.404743 1172 ts_tablet_manager.cc:1434] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260505 07:54:15.405009 1181 tablet_bootstrap.cc:654] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.405037 1172 tablet_bootstrap.cc:492] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.407009 2405 raft_consensus.cc:493] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.407161 2405 raft_consensus.cc:515] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.407882 2405 leader_election.cc:290] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.408354 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e06d8815034d91986cef12964860ea" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.408535 775 raft_consensus.cc:2393] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:15.408599 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02e06d8815034d91986cef12964860ea" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.409341 991 leader_election.cc:304] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.409509 1052 raft_consensus.cc:3060] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.409833 2405 raft_consensus.cc:3060] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.412199 1172 tablet_bootstrap.cc:654] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.412201 2405 raft_consensus.cc:2749] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.403781 2268 rpcz_store.cc:276] 0505 07:54:09.818963 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.819676 (+ 713us) service_pool.cc:224] Handling call
0505 07:54:15.325094 (+5505418us) ts_tablet_manager.cc:1151] Deleting tablet 93d878ae0ddb42319560f74e0a490321
0505 07:54:15.329193 (+ 4099us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.364350 (+ 35157us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.376374 (+ 12024us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.397252 (+ 20878us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5505275}
I20260505 07:54:15.412698 2268 tablet_replica.cc:333] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.413141 2268 raft_consensus.cc:2243] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.413374 2872 consensus_queue.cc:1048] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.414932 2268 pending_rounds.cc:70] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:54:15.415370 2872 raft_consensus.cc:493] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.415477 2872 raft_consensus.cc:515] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.416019 2872 leader_election.cc:290] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.416677 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77ff90ec2ff4b6eb1795d149459c23a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.416854 920 raft_consensus.cc:2468] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:15.416277 2405 consensus_queue.cc:1048] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.416911 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77ff90ec2ff4b6eb1795d149459c23a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.417282 718 leader_election.cc:304] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.417577 2674 raft_consensus.cc:2804] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:15.417671 2674 raft_consensus.cc:493] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:15.417742 2674 raft_consensus.cc:3060] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.420480 2674 raft_consensus.cc:515] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:15.421574 717 leader_election.cc:343] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.421973 2872 raft_consensus.cc:493] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.422103 2872 raft_consensus.cc:515] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.422613 2872 leader_election.cc:290] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.423309 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77ff90ec2ff4b6eb1795d149459c23a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:15.423382 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec145bdc0dab4642ba233bd0da2c436f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.423447 916 raft_consensus.cc:3060] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.423487 918 raft_consensus.cc:2468] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:15.424686 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec145bdc0dab4642ba233bd0da2c436f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.424861 1051 raft_consensus.cc:2468] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:15.424912 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d77ff90ec2ff4b6eb1795d149459c23a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:15.425211 1172 tablet_bootstrap.cc:492] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.425324 1172 ts_tablet_manager.cc:1403] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.003s
I20260505 07:54:15.425359 2674 leader_election.cc:290] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.425671 718 leader_election.cc:304] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.425940 2742 consensus_queue.cc:1048] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.425990 1172 raft_consensus.cc:359] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.426134 1172 raft_consensus.cc:385] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.426187 1172 raft_consensus.cc:740] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:54:15.426250 717 leader_election.cc:343] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.426193 916 raft_consensus.cc:2468] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:15.426617 718 leader_election.cc:304] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:15.426633 2872 raft_consensus.cc:2804] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:15.426765 2872 raft_consensus.cc:493] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:15.426836 2872 raft_consensus.cc:3060] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.426419 1172 consensus_queue.cc:260] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.427481 1172 ts_tablet_manager.cc:1434] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:15.427816 1172 tablet_bootstrap.cc:492] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.428865 2831 raft_consensus.cc:2804] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:15.428984 2831 raft_consensus.cc:697] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:15.429229 2831 consensus_queue.cc:237] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.429771 1172 tablet_bootstrap.cc:654] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.431790 2872 raft_consensus.cc:515] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.438098 2268 pending_rounds.cc:77] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7282555513045630976 op_type: NO_OP noop_request { }
I20260505 07:54:15.438235 2268 raft_consensus.cc:2889] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:15.438335 2268 raft_consensus.cc:2272] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.439052 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec145bdc0dab4642ba233bd0da2c436f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:15.439191 918 raft_consensus.cc:3060] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.439189 2670 raft_consensus.cc:493] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.439287 2670 raft_consensus.cc:515] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.439778 2670 leader_election.cc:290] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.440374 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.440492 907 raft_consensus.cc:2468] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:15.440841 718 leader_election.cc:304] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.441068 2670 raft_consensus.cc:2804] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:15.441119 2670 raft_consensus.cc:493] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:15.441164 2670 raft_consensus.cc:3060] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.441272 2268 ts_tablet_manager.cc:1916] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.441685 918 raft_consensus.cc:2468] T ec145bdc0dab4642ba233bd0da2c436f P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:15.443753 2670 raft_consensus.cc:515] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.444788 718 leader_election.cc:304] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.445256 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:15.445399 912 raft_consensus.cc:3060] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.445796 2670 leader_election.cc:290] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.446393 2670 raft_consensus.cc:2804] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:15.446533 2670 raft_consensus.cc:697] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:15.446779 2670 consensus_queue.cc:237] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.391801 2276 ts_tablet_manager.cc:1929] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.447659 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:15.447727 2276 log.cc:1199] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/b0b9e8eb767a4eceb583ff9b47000d58
I20260505 07:54:15.447804 912 raft_consensus.cc:2468] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:15.447997 2752 raft_consensus.cc:493] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.448060 2276 ts_tablet_manager.cc:1950] T b0b9e8eb767a4eceb583ff9b47000d58 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.448127 2752 raft_consensus.cc:515] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.448316 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 121) took 7665 ms. Trace:
I20260505 07:54:15.448683 2752 leader_election.cc:290] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.450230 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225c56ac920b448c89c38d6fb3c1bdef" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:15.450771 856 leader_election.cc:343] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.448377 2276 rpcz_store.cc:276] 0505 07:54:07.782943 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.800708 (+ 17765us) service_pool.cc:224] Handling call
0505 07:54:15.344031 (+7543323us) ts_tablet_manager.cc:1151] Deleting tablet b0b9e8eb767a4eceb583ff9b47000d58
0505 07:54:15.344040 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.379121 (+ 35081us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.391756 (+ 12635us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.448302 (+ 56546us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7543159}
I20260505 07:54:15.451747 2276 tablet_replica.cc:333] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.451948 2276 raft_consensus.cc:2243] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.452162 2276 pending_rounds.cc:70] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:54:15.452219 2276 pending_rounds.cc:77] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7282555514866974720 op_type: NO_OP noop_request { }
I20260505 07:54:15.452299 2276 raft_consensus.cc:2889] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:15.452380 2276 raft_consensus.cc:2272] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.452760 1172 tablet_bootstrap.cc:492] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.452951 1172 ts_tablet_manager.cc:1403] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.005s
I20260505 07:54:15.453164 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 93d878ae0ddb42319560f74e0a490321 (table default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]) successfully deleted
I20260505 07:54:15.453594 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet b0b9e8eb767a4eceb583ff9b47000d58 (table default.Table_49c6ff5135b14b6b8437bb753e1b6342 [id=169b57b0e527456cb435bd97c47782bb]) successfully deleted
I20260505 07:54:15.453701 1172 raft_consensus.cc:359] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.453830 1172 raft_consensus.cc:385] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.453878 1172 raft_consensus.cc:740] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.454097 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:15.454110 1172 consensus_queue.cc:260] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.454576 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:15.454669 1172 ts_tablet_manager.cc:1434] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:15.454985 1172 tablet_bootstrap.cc:492] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.455204 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:15.447443 2447 ts_tablet_manager.cc:1929] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260505 07:54:15.449033 717 leader_election.cc:343] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.450700 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225c56ac920b448c89c38d6fb3c1bdef" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.456553 1172 tablet_bootstrap.cc:654] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:15.456700 854 leader_election.cc:343] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.456812 854 leader_election.cc:304] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.457000 2752 raft_consensus.cc:2749] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.457691 2872 leader_election.cc:290] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.458904 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec145bdc0dab4642ba233bd0da2c436f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:15.459074 1053 raft_consensus.cc:3060] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.461576 1053 raft_consensus.cc:2468] T ec145bdc0dab4642ba233bd0da2c436f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:15.462162 2447 log.cc:1199] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/78a6470720984761a8606abeafbef992
I20260505 07:54:15.462523 2447 ts_tablet_manager.cc:1950] T 78a6470720984761a8606abeafbef992 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:15.463184 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 167) took 5913 ms. Trace:
I20260505 07:54:15.463263 2447 rpcz_store.cc:276] 0505 07:54:09.549405 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.549459 (+ 54us) service_pool.cc:224] Handling call
0505 07:54:15.347392 (+5797933us) ts_tablet_manager.cc:1151] Deleting tablet 78a6470720984761a8606abeafbef992
0505 07:54:15.349123 (+ 1731us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.403224 (+ 54101us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.447399 (+ 44175us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.463167 (+ 15768us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":453,"tablet-delete.queue_time_us":5789111}
I20260505 07:54:15.463629 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 78a6470720984761a8606abeafbef992 (table default.Table_5cd903011cdc44a392680507ed9088c8 [id=a3c321e46ecd4b07809a9c663305afb7]) successfully deleted
I20260505 07:54:15.464056 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:15.464663 2276 ts_tablet_manager.cc:1916] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.465207 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:15.466886 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:15.475268 1172 tablet_bootstrap.cc:492] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.475390 1172 ts_tablet_manager.cc:1403] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.000s
I20260505 07:54:15.473841 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:15.475950 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:15.476123 1172 raft_consensus.cc:359] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.476269 1172 raft_consensus.cc:385] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.476321 1172 raft_consensus.cc:740] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.476492 1172 consensus_queue.cc:260] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.476610 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 169) took 5926 ms. Trace:
I20260505 07:54:15.477195 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:15.477496 1172 ts_tablet_manager.cc:1434] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:15.477859 1172 tablet_bootstrap.cc:492] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.477856 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
W20260505 07:54:15.478298 991 consensus_peers.cc:597] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a7d8265c05b74b28959d0de41fad1fde. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:15.478413 1349 catalog_manager.cc:5671] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:15.479449 1172 tablet_bootstrap.cc:654] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.484687 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:15.485148 717 leader_election.cc:343] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.486668 1349 catalog_manager.cc:5671] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:15.487828 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:15.488598 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:15.489595 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:15.492297 2268 ts_tablet_manager.cc:1929] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:15.497828 1172 tablet_bootstrap.cc:492] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.498005 1172 ts_tablet_manager.cc:1403] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260505 07:54:15.500864 2872 consensus_queue.cc:1048] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.477386 2447 rpcz_store.cc:276] 0505 07:54:09.549836 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.549899 (+ 63us) service_pool.cc:224] Handling call
0505 07:54:15.476481 (+5926582us) ts_tablet_manager.cc:1151] Deleting tablet e5eaa91ecc1f43d18b313b6be4cc211e
0505 07:54:15.476490 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.476594 (+ 104us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5917692}
I20260505 07:54:15.502180 1172 raft_consensus.cc:359] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.502327 1172 raft_consensus.cc:385] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.502388 1172 raft_consensus.cc:740] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.502614 1172 consensus_queue.cc:260] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.503095 1172 ts_tablet_manager.cc:1434] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260505 07:54:15.503374 1172 tablet_bootstrap.cc:492] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.505014 1172 tablet_bootstrap.cc:654] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:15.507566 856 consensus_peers.cc:597] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0459f2d45a2d44f1a73e0a0fff53866c. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:15.508981 2268 log.cc:1199] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/9a06f77255544d7dabb71a4f70ada809
W20260505 07:54:15.502151 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e5eaa91ecc1f43d18b313b6be4cc211e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5eaa91ecc1f43d18b313b6be4cc211e already in progress: creating tablet
I20260505 07:54:15.516045 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:15.520218 1176 tablet_bootstrap.cc:492] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.520396 1176 ts_tablet_manager.cc:1403] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.153s user 0.005s sys 0.001s
I20260505 07:54:15.493532 1181 tablet_bootstrap.cc:492] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.520736 1181 ts_tablet_manager.cc:1403] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.127s user 0.000s sys 0.006s
I20260505 07:54:15.520776 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:15.521289 2447 tablet_replica.cc:333] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:15.521938 2447 raft_consensus.cc:2243] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.521912 1181 raft_consensus.cc:359] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.522189 1181 raft_consensus.cc:385] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.522289 1181 raft_consensus.cc:740] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.522539 1181 consensus_queue.cc:260] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.523073 1181 ts_tablet_manager.cc:1434] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:15.523435 1181 tablet_bootstrap.cc:492] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.523384 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:15.523454 1176 raft_consensus.cc:359] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.523659 1176 raft_consensus.cc:385] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.523739 1176 raft_consensus.cc:740] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.523885 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:15.523937 1176 consensus_queue.cc:260] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.524602 1176 ts_tablet_manager.cc:1434] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:15.524895 1176 tablet_bootstrap.cc:492] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.525022 1181 tablet_bootstrap.cc:654] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.526116 2268 ts_tablet_manager.cc:1950] T 9a06f77255544d7dabb71a4f70ada809 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.526494 2447 raft_consensus.cc:2272] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.527145 2447 ts_tablet_manager.cc:1916] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.528542 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 153) took 5708 ms. Trace:
I20260505 07:54:15.528990 2268 rpcz_store.cc:276] 0505 07:54:09.820087 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.825584 (+ 5497us) service_pool.cc:224] Handling call
0505 07:54:15.412632 (+5587048us) ts_tablet_manager.cc:1151] Deleting tablet 9a06f77255544d7dabb71a4f70ada809
0505 07:54:15.412641 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.443959 (+ 31318us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.492251 (+ 48292us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.528526 (+ 36275us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5586872}
I20260505 07:54:15.529992 2268 tablet_replica.cc:333] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.530584 2268 raft_consensus.cc:2243] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.531677 2268 pending_rounds.cc:70] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:54:15.531986 2268 pending_rounds.cc:77] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7282555515847372800 op_type: NO_OP noop_request { }
I20260505 07:54:15.532297 2268 raft_consensus.cc:2889] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:15.532567 2268 raft_consensus.cc:2272] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.531757 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 9a06f77255544d7dabb71a4f70ada809 (table default.Table_8091d8a01cbf47f3b270874cd5e6140e [id=2a8cda02af654aa5b0c161e04968c735]) successfully deleted
I20260505 07:54:15.534950 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:15.535398 1181 tablet_bootstrap.cc:492] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.535533 1181 ts_tablet_manager.cc:1403] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.004s
I20260505 07:54:15.536041 1176 tablet_bootstrap.cc:654] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.536288 1181 raft_consensus.cc:359] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.536465 1181 raft_consensus.cc:385] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.536550 1181 raft_consensus.cc:740] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.536748 1181 consensus_queue.cc:260] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.537240 2276 ts_tablet_manager.cc:1929] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:15.537336 2276 log.cc:1199] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/727cacd242844455915bd9963d2db19f
I20260505 07:54:15.537658 2276 ts_tablet_manager.cc:1950] T 727cacd242844455915bd9963d2db19f P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.537940 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 122) took 7754 ms. Trace:
I20260505 07:54:15.538013 2276 rpcz_store.cc:276] 0505 07:54:07.783015 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.800884 (+ 17869us) service_pool.cc:224] Handling call
0505 07:54:15.451703 (+7650819us) ts_tablet_manager.cc:1151] Deleting tablet 727cacd242844455915bd9963d2db19f
0505 07:54:15.451714 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.485965 (+ 34251us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.537185 (+ 51220us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.537926 (+ 741us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7650681}
I20260505 07:54:15.538298 2276 tablet_replica.cc:333] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.538480 2276 raft_consensus.cc:2243] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.538611 1181 ts_tablet_manager.cc:1434] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:15.538621 2276 raft_consensus.cc:2272] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.538635 3004 raft_consensus.cc:493] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.538765 3004 raft_consensus.cc:515] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.539113 1181 tablet_bootstrap.cc:492] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.539127 2276 ts_tablet_manager.cc:1916] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.539800 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3a49b59ed6485f970074391f932482" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.539960 1048 raft_consensus.cc:2468] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:15.540339 717 leader_election.cc:304] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.540432 1181 tablet_bootstrap.cc:654] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.540567 2872 raft_consensus.cc:2804] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:15.540633 2872 raft_consensus.cc:493] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:15.540699 2872 raft_consensus.cc:3060] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.540892 3004 leader_election.cc:290] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:15.541096 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 96321.6us
I20260505 07:54:15.541422 718 leader_election.cc:304] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:15.541676 3004 consensus_queue.cc:1048] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.541785 1172 tablet_bootstrap.cc:492] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.541880 1172 ts_tablet_manager.cc:1403] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.039s user 0.003s sys 0.002s
I20260505 07:54:15.542183 3004 raft_consensus.cc:2804] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:15.542268 3004 raft_consensus.cc:697] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:15.542495 1172 raft_consensus.cc:359] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.542646 1172 raft_consensus.cc:385] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.542685 1172 raft_consensus.cc:740] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.542866 1172 consensus_queue.cc:260] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.543111 2872 raft_consensus.cc:515] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.543871 2872 leader_election.cc:290] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.544306 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3a49b59ed6485f970074391f932482" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:15.544435 1048 raft_consensus.cc:3060] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.545295 2268 ts_tablet_manager.cc:1916] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.547314 1048 raft_consensus.cc:2468] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:15.547639 717 leader_election.cc:304] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.548041 2872 raft_consensus.cc:2804] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:15.548151 1172 ts_tablet_manager.cc:1434] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260505 07:54:15.548295 2872 raft_consensus.cc:697] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:15.548395 1172 tablet_bootstrap.cc:492] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.548635 2872 consensus_queue.cc:237] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.538463 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 727cacd242844455915bd9963d2db19f (table default.Table_66ccaae918c545228101f7d0e29c37dc [id=ad11a4d7ee734c9bab8278aebac5cb40]) successfully deleted
I20260505 07:54:15.552276 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:15.556473 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:15.557288 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:15.542444 3004 consensus_queue.cc:237] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.562789 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:15.563752 2752 consensus_queue.cc:1048] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.563887 1172 tablet_bootstrap.cc:654] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.565513 1181 tablet_bootstrap.cc:492] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.565718 1181 ts_tablet_manager.cc:1403] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.000s
I20260505 07:54:15.566964 1181 raft_consensus.cc:359] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.567830 1181 raft_consensus.cc:385] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.567957 1181 raft_consensus.cc:740] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.568208 1181 consensus_queue.cc:260] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.569168 1181 ts_tablet_manager.cc:1434] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:15.569734 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:15.570502 1181 tablet_bootstrap.cc:492] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.574692 1181 tablet_bootstrap.cc:654] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.576277 1172 tablet_bootstrap.cc:492] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.576449 1172 ts_tablet_manager.cc:1403] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.001s
I20260505 07:54:15.577241 1172 raft_consensus.cc:359] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.577452 1172 raft_consensus.cc:385] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.577558 1172 raft_consensus.cc:740] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.577804 1172 consensus_queue.cc:260] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.578336 1172 ts_tablet_manager.cc:1434] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20260505 07:54:15.579533 991 consensus_peers.cc:597] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3075e87e078f4d2aa1d329d18902354b. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:15.580590 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:15.582314 1172 tablet_bootstrap.cc:492] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.584103 2752 consensus_queue.cc:1048] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.584679 2752 raft_consensus.cc:493] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.584797 2752 raft_consensus.cc:515] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.585420 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:15.585812 2752 leader_election.cc:290] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.586421 1172 tablet_bootstrap.cc:654] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.586359 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3a49b59ed6485f970074391f932482" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.586612 1050 raft_consensus.cc:2393] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d2799f5c6b4f4336b976ca9bbb458533 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:15.590091 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:15.590868 1181 tablet_bootstrap.cc:492] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.591027 1181 ts_tablet_manager.cc:1403] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20260505 07:54:15.591749 1181 raft_consensus.cc:359] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.591948 1181 raft_consensus.cc:385] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.592034 1181 raft_consensus.cc:740] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.592290 1181 consensus_queue.cc:260] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.594306 1181 ts_tablet_manager.cc:1434] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:15.595619 1181 tablet_bootstrap.cc:492] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.592587 2268 ts_tablet_manager.cc:1929] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:15.596323 2268 log.cc:1199] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/cac5064f68d24ebe95b5a0b8e10f463e
I20260505 07:54:15.596809 2268 ts_tablet_manager.cc:1950] T cac5064f68d24ebe95b5a0b8e10f463e P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.597885 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 155) took 5755 ms. Trace:
I20260505 07:54:15.599789 2268 rpcz_store.cc:276] 0505 07:54:09.842794 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.845201 (+ 2407us) service_pool.cc:224] Handling call
0505 07:54:15.529940 (+5684739us) ts_tablet_manager.cc:1151] Deleting tablet cac5064f68d24ebe95b5a0b8e10f463e
0505 07:54:15.529950 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.558902 (+ 28952us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.592536 (+ 33634us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.597868 (+ 5332us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5684544}
I20260505 07:54:15.597162 1181 tablet_bootstrap.cc:654] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.602087 2447 ts_tablet_manager.cc:1929] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.602859 1172 tablet_bootstrap.cc:492] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.602973 1172 ts_tablet_manager.cc:1403] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.003s
I20260505 07:54:15.603677 1172 raft_consensus.cc:359] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.603885 1172 raft_consensus.cc:385] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.603937 1172 raft_consensus.cc:740] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.600569 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet cac5064f68d24ebe95b5a0b8e10f463e (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]) successfully deleted
I20260505 07:54:15.604324 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:15.604287 1172 consensus_queue.cc:260] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.604538 2268 tablet_replica.cc:333] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.604895 1172 ts_tablet_manager.cc:1434] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:15.605168 2268 raft_consensus.cc:2243] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.605186 1172 tablet_bootstrap.cc:492] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.606072 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:15.606719 1172 tablet_bootstrap.cc:654] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.607048 2268 raft_consensus.cc:2272] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.607676 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:15.610492 2268 ts_tablet_manager.cc:1916] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.616276 1172 tablet_bootstrap.cc:492] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.616392 1172 ts_tablet_manager.cc:1403] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
I20260505 07:54:15.617079 1172 raft_consensus.cc:359] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.617242 1172 raft_consensus.cc:385] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.617807 1172 raft_consensus.cc:740] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.618052 1172 consensus_queue.cc:260] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.618294 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:15.619411 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:15.620383 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:15.622421 1172 ts_tablet_manager.cc:1434] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260505 07:54:15.622664 1172 tablet_bootstrap.cc:492] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.626392 2276 ts_tablet_manager.cc:1929] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.631719 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:15.605201 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3a49b59ed6485f970074391f932482" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.631561 1176 tablet_bootstrap.cc:492] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.634537 1176 ts_tablet_manager.cc:1403] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.110s user 0.006s sys 0.009s
I20260505 07:54:15.635186 1176 raft_consensus.cc:359] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.635358 1176 raft_consensus.cc:385] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.635414 1176 raft_consensus.cc:740] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.635411 2268 ts_tablet_manager.cc:1929] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:15.635497 2268 log.cc:1199] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/d77ff90ec2ff4b6eb1795d149459c23a
I20260505 07:54:15.635717 1176 consensus_queue.cc:260] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.636229 1176 ts_tablet_manager.cc:1434] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:15.636492 1176 tablet_bootstrap.cc:492] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.638553 1052 raft_consensus.cc:1275] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Refusing update from remote peer eac7e1e8caba43cdb4df80cbb80e891c: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260505 07:54:15.638906 2447 log.cc:1199] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/03f31efbc98c484c950c70d5e84d2136
I20260505 07:54:15.639326 2447 ts_tablet_manager.cc:1950] T 03f31efbc98c484c950c70d5e84d2136 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:15.640542 1181 tablet_bootstrap.cc:492] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.640690 1181 ts_tablet_manager.cc:1403] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.045s user 0.007s sys 0.000s
I20260505 07:54:15.641320 1181 raft_consensus.cc:359] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.641463 1181 raft_consensus.cc:385] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.641551 1181 raft_consensus.cc:740] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.641552 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 191) took 6085 ms. Trace:
I20260505 07:54:15.641616 2447 rpcz_store.cc:276] 0505 07:54:09.555690 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.558602 (+ 2912us) service_pool.cc:224] Handling call
0505 07:54:15.502414 (+5943812us) ts_tablet_manager.cc:1151] Deleting tablet 03f31efbc98c484c950c70d5e84d2136
0505 07:54:15.521223 (+ 18809us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.548940 (+ 27717us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.602036 (+ 53096us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.641534 (+ 39498us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5943513}
I20260505 07:54:15.641727 1181 consensus_queue.cc:260] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.639892 1176 tablet_bootstrap.cc:654] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.642218 1181 ts_tablet_manager.cc:1434] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260505 07:54:15.642285 2872 consensus_queue.cc:1048] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260505 07:54:15.642454 1181 tablet_bootstrap.cc:492] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
W20260505 07:54:15.644019 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 102605us
I20260505 07:54:15.644438 1181 tablet_bootstrap.cc:654] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.646505 2268 ts_tablet_manager.cc:1950] T d77ff90ec2ff4b6eb1795d149459c23a P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.647490 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 156) took 5801 ms. Trace:
I20260505 07:54:15.648816 2268 rpcz_store.cc:276] 0505 07:54:09.845647 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.846758 (+ 1111us) service_pool.cc:224] Handling call
0505 07:54:15.604457 (+5757699us) ts_tablet_manager.cc:1151] Deleting tablet d77ff90ec2ff4b6eb1795d149459c23a
0505 07:54:15.604467 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.619388 (+ 14921us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.635364 (+ 15976us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.647473 (+ 12109us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5757407}
I20260505 07:54:15.650944 2268 tablet_replica.cc:333] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.651158 2268 raft_consensus.cc:2243] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.651345 2268 raft_consensus.cc:2272] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.651875 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 171) took 6101 ms. Trace:
I20260505 07:54:15.651991 2447 rpcz_store.cc:276] 0505 07:54:09.550621 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.550679 (+ 58us) service_pool.cc:224] Handling call
0505 07:54:15.651771 (+6101092us) ts_tablet_manager.cc:1151] Deleting tablet abaa596871bb4a82aefc317932e91d35
0505 07:54:15.651778 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.651861 (+ 83us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6092945}
I20260505 07:54:15.652431 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 172) took 6101 ms. Trace:
I20260505 07:54:15.652542 2447 rpcz_store.cc:276] 0505 07:54:09.551307 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.551353 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:15.652342 (+6100989us) ts_tablet_manager.cc:1151] Deleting tablet 72da22d790624b07bfca6f6b6301c925
0505 07:54:15.652352 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.652418 (+ 66us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6093449}
I20260505 07:54:15.652920 2447 tablet_replica.cc:333] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:15.653275 2447 raft_consensus.cc:2243] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.653576 2447 raft_consensus.cc:2272] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.642011 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 03f31efbc98c484c950c70d5e84d2136 (table default.Table_6ff000247d324fe8b55b1398e48cb49a [id=8903351259c347a1925769663d34d173]) successfully deleted
I20260505 07:54:15.653846 1878 raft_consensus.cc:493] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.653973 1878 raft_consensus.cc:515] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:15.654081 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet abaa596871bb4a82aefc317932e91d35 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abaa596871bb4a82aefc317932e91d35 already in progress: creating tablet
I20260505 07:54:15.654146 2268 ts_tablet_manager.cc:1916] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.654146 2447 ts_tablet_manager.cc:1916] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:15.654317 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 72da22d790624b07bfca6f6b6301c925 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 72da22d790624b07bfca6f6b6301c925 already in progress: creating tablet
I20260505 07:54:15.650923 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet d77ff90ec2ff4b6eb1795d149459c23a (table default.Table_1ff5286907c6464c8addf47c70eef7ef [id=f608e93cbc64485297c6dc2609a6eb87]) successfully deleted
I20260505 07:54:15.654541 1878 leader_election.cc:290] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.654873 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:15.655122 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:15.655457 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e4e5af08ddc4c77b5640b8295b236b1 (table default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09]). Expected version 1 got 0
I20260505 07:54:15.661327 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:15.627363 2752 raft_consensus.cc:493] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.630155 1172 tablet_bootstrap.cc:654] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.669776 2276 log.cc:1199] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/6ca2852cee84476dbe5e5072a43087a1
I20260505 07:54:15.669750 2752 raft_consensus.cc:515] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.670202 2276 ts_tablet_manager.cc:1950] T 6ca2852cee84476dbe5e5072a43087a1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.670446 2752 leader_election.cc:290] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.634536 854 leader_election.cc:304] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.670921 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 123) took 7887 ms. Trace:
I20260505 07:54:15.671026 2276 rpcz_store.cc:276] 0505 07:54:07.783094 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.801009 (+ 17915us) service_pool.cc:224] Handling call
0505 07:54:15.538263 (+7737254us) ts_tablet_manager.cc:1151] Deleting tablet 6ca2852cee84476dbe5e5072a43087a1
0505 07:54:15.538271 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.595045 (+ 56774us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.626350 (+ 31305us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.670905 (+ 44555us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7737128}
I20260505 07:54:15.671126 2505 raft_consensus.cc:3060] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.671293 2276 tablet_replica.cc:333] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.671474 2276 raft_consensus.cc:2243] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.671633 2276 raft_consensus.cc:2272] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.671823 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3a49b59ed6485f970074391f932482" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:15.637840 2856 raft_consensus.cc:493] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.654929 916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02716f62aa354d8f839266661b7f05ea" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.672417 916 raft_consensus.cc:2468] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:15.672395 2856 raft_consensus.cc:515] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.672866 2856 leader_election.cc:290] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.672899 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02716f62aa354d8f839266661b7f05ea" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.673046 775 raft_consensus.cc:2468] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:15.673532 2505 raft_consensus.cc:2749] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.674408 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df3a49b59ed6485f970074391f932482" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.672626 2276 ts_tablet_manager.cc:1916] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.674985 991 leader_election.cc:304] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:15.674959 2670 consensus_queue.cc:1048] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.675243 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac029d9a3c540f58f393e79bf7be30e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.675423 2302 raft_consensus.cc:2804] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:15.675487 2302 raft_consensus.cc:493] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:15.675541 2302 raft_consensus.cc:3060] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:15.677029 854 leader_election.cc:343] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.677740 2302 raft_consensus.cc:515] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.678180 2302 leader_election.cc:290] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.678344 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235a3ccef904913bbd67c403aac3a87" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.679620 1048 raft_consensus.cc:3060] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.679723 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235a3ccef904913bbd67c403aac3a87" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.655989 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:15.680053 2650 consensus_queue.cc:1048] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.680249 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
W20260505 07:54:15.680452 854 leader_election.cc:343] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.680565 2650 consensus_queue.cc:1048] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.680714 2856 consensus_queue.cc:1048] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.680837 2650 consensus_queue.cc:1048] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.681033 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:15.681370 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:15.678640 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 6ca2852cee84476dbe5e5072a43087a1 (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]) successfully deleted
I20260505 07:54:15.681569 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:15.681897 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:15.682152 1349 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:15.682561 2670 consensus_queue.cc:1048] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.684355 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02716f62aa354d8f839266661b7f05ea" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:15.684504 911 raft_consensus.cc:3060] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.685642 1176 tablet_bootstrap.cc:492] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.686076 1176 ts_tablet_manager.cc:1403] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.050s user 0.002s sys 0.001s
I20260505 07:54:15.686869 911 raft_consensus.cc:2468] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:15.687942 991 leader_election.cc:304] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:15.687528 1176 raft_consensus.cc:359] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.688234 2948 raft_consensus.cc:2804] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:15.688290 1176 raft_consensus.cc:385] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.688330 2948 raft_consensus.cc:697] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:15.688346 1176 raft_consensus.cc:740] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.688529 1176 consensus_queue.cc:260] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.688545 2948 consensus_queue.cc:237] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:15.690443 854 consensus_peers.cc:597] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0459f2d45a2d44f1a73e0a0fff53866c. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:15.690609 2856 raft_consensus.cc:493] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.690706 2856 raft_consensus.cc:515] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.691167 2856 leader_election.cc:290] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.691535 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8e6019406845ebb29051432a96dfd1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:15.700038 854 leader_election.cc:343] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.689615 1176 ts_tablet_manager.cc:1434] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:15.700475 1176 tablet_bootstrap.cc:492] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.702556 920 raft_consensus.cc:2468] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:15.718987 1354 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 194) took 2595 ms. Trace:
I20260505 07:54:15.719158 1354 rpcz_store.cc:276] 0505 07:54:13.123834 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.290691 (+2166857us) service_pool.cc:224] Handling call
0505 07:54:15.718930 (+428239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260505 07:54:15.726243 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac029d9a3c540f58f393e79bf7be30e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.730491 1176 tablet_bootstrap.cc:654] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.732152 1181 tablet_bootstrap.cc:492] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.732378 1181 ts_tablet_manager.cc:1403] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.090s user 0.001s sys 0.006s
I20260505 07:54:15.733104 1181 raft_consensus.cc:359] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.733376 1181 raft_consensus.cc:385] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.733472 1181 raft_consensus.cc:740] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.733757 1181 consensus_queue.cc:260] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:15.737525 991 consensus_peers.cc:597] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d622612bdc404727a27b83769bf39738. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:15.740327 1172 tablet_bootstrap.cc:492] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.740450 1172 ts_tablet_manager.cc:1403] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.118s user 0.004s sys 0.008s
I20260505 07:54:15.741353 1172 raft_consensus.cc:359] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.741509 1172 raft_consensus.cc:385] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.741562 1172 raft_consensus.cc:740] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.741755 1172 consensus_queue.cc:260] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.742228 1172 ts_tablet_manager.cc:1434] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:15.742461 1172 tablet_bootstrap.cc:492] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.742974 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02716f62aa354d8f839266661b7f05ea" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:15.743310 772 raft_consensus.cc:3060] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.743738 2670 consensus_queue.cc:1048] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.746506 2447 ts_tablet_manager.cc:1929] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.747682 2276 ts_tablet_manager.cc:1929] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.747785 2276 log.cc:1199] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/297537a92684425d8508f7b3b4c9d379
I20260505 07:54:15.748694 1181 ts_tablet_manager.cc:1434] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.016s user 0.001s sys 0.000s
I20260505 07:54:15.747702 2872 consensus_queue.cc:1048] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.749138 1181 tablet_bootstrap.cc:492] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.748047 3013 consensus_queue.cc:1048] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.749276 1172 tablet_bootstrap.cc:654] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:15.748488 718 consensus_peers.cc:597] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff7ed926a3ff4e7993864de734ca3c50. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:15.748790 2670 raft_consensus.cc:493] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.749656 2670 raft_consensus.cc:515] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.750224 2670 leader_election.cc:290] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.750691 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9963a0f1a54ee9a73486bb1801ee74" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.750849 1048 raft_consensus.cc:2468] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:15.751255 717 leader_election.cc:304] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.750849 1181 tablet_bootstrap.cc:654] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.751756 2872 raft_consensus.cc:2804] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:15.751880 2872 raft_consensus.cc:493] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:15.751992 2872 raft_consensus.cc:3060] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.754467 2872 raft_consensus.cc:515] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:15.754845 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 75322.4us
I20260505 07:54:15.755475 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9963a0f1a54ee9a73486bb1801ee74" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:15.755625 1048 raft_consensus.cc:3060] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.758035 1048 raft_consensus.cc:2468] T 4b9963a0f1a54ee9a73486bb1801ee74 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:15.758401 856 leader_election.cc:343] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.758508 856 leader_election.cc:304] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.758535 717 leader_election.cc:304] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:15.758766 3013 raft_consensus.cc:2804] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:15.758904 3013 raft_consensus.cc:697] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
W20260505 07:54:15.758888 856 leader_election.cc:343] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.758991 856 leader_election.cc:304] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.759138 3013 consensus_queue.cc:237] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.759286 2276 ts_tablet_manager.cc:1950] T 297537a92684425d8508f7b3b4c9d379 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.759625 2650 raft_consensus.cc:493] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.759764 2752 raft_consensus.cc:2749] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.759732 2650 raft_consensus.cc:515] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.759891 2752 raft_consensus.cc:2749] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.760027 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 124) took 7976 ms. Trace:
I20260505 07:54:15.760106 2276 rpcz_store.cc:276] 0505 07:54:07.783171 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.801147 (+ 17976us) service_pool.cc:224] Handling call
0505 07:54:15.671255 (+7870108us) ts_tablet_manager.cc:1151] Deleting tablet 297537a92684425d8508f7b3b4c9d379
0505 07:54:15.671263 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.677884 (+ 6621us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.747615 (+ 69731us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.760014 (+ 12399us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7869958}
I20260505 07:54:15.760385 2276 tablet_replica.cc:333] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.760531 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0d43d85fcd4d05abf10d415e8dafbe" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.760962 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 297537a92684425d8508f7b3b4c9d379 (table default.Table_44ead87f92584e6e98e2c5f69056591a [id=e78409bbbd334198b96cc757bc1feee8]) successfully deleted
I20260505 07:54:15.761662 2650 leader_election.cc:290] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.763332 2268 ts_tablet_manager.cc:1929] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.763484 2268 log.cc:1199] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/c1a61c1d03d544c8aa01537ead97f83c
I20260505 07:54:15.763940 2268 ts_tablet_manager.cc:1950] T c1a61c1d03d544c8aa01537ead97f83c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.765460 1048 raft_consensus.cc:3060] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.766652 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8e6019406845ebb29051432a96dfd1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:15.767550 856 leader_election.cc:343] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.767663 856 leader_election.cc:304] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.768034 2505 raft_consensus.cc:2749] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.769985 2505 consensus_queue.cc:1048] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.771081 1181 tablet_bootstrap.cc:492] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.773521 1172 tablet_bootstrap.cc:492] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.773645 1172 ts_tablet_manager.cc:1403] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.031s user 0.002s sys 0.000s
I20260505 07:54:15.773766 1181 ts_tablet_manager.cc:1403] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.025s user 0.003s sys 0.000s
I20260505 07:54:15.774276 1172 raft_consensus.cc:359] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.774435 1172 raft_consensus.cc:385] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.774485 1172 raft_consensus.cc:740] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.774708 1172 consensus_queue.cc:260] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.775171 1172 ts_tablet_manager.cc:1434] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260505 07:54:15.775404 1172 tablet_bootstrap.cc:492] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.775398 1181 raft_consensus.cc:359] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.775705 1181 raft_consensus.cc:385] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.775811 1181 raft_consensus.cc:740] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.776180 1181 consensus_queue.cc:260] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.776706 1172 tablet_bootstrap.cc:654] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.776979 1181 ts_tablet_manager.cc:1434] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:15.781351 2505 consensus_queue.cc:1048] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.789556 1181 tablet_bootstrap.cc:492] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.789536 2872 leader_election.cc:290] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.791337 1181 tablet_bootstrap.cc:654] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.792495 2505 raft_consensus.cc:493] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.792605 2505 raft_consensus.cc:515] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.793205 2505 leader_election.cc:290] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.793531 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09019fdf60374b11bf3427f21a55b5e0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.793664 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09019fdf60374b11bf3427f21a55b5e0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.793835 2505 consensus_queue.cc:1048] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:15.794103 854 leader_election.cc:343] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:15.794351 856 leader_election.cc:343] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.794487 856 leader_election.cc:304] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.794883 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 157) took 5947 ms. Trace:
I20260505 07:54:15.794977 2268 rpcz_store.cc:276] 0505 07:54:09.847048 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.847405 (+ 357us) service_pool.cc:224] Handling call
0505 07:54:15.650894 (+5803489us) ts_tablet_manager.cc:1151] Deleting tablet c1a61c1d03d544c8aa01537ead97f83c
0505 07:54:15.650903 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.657227 (+ 6324us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.763269 (+106042us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.794859 (+ 31590us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5803334}
I20260505 07:54:15.795318 2505 raft_consensus.cc:2749] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.795631 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet c1a61c1d03d544c8aa01537ead97f83c (table default.Table_575ab5b128ee4a1baebbe1f90baf005a [id=cdb574c7a4e14a5ea7f3c8b036a8c336]) successfully deleted
I20260505 07:54:15.796137 2268 tablet_replica.cc:333] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.796440 2268 raft_consensus.cc:2243] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.796752 2268 pending_rounds.cc:70] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:54:15.796842 2268 pending_rounds.cc:77] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555517702848512 op_type: NO_OP noop_request { }
I20260505 07:54:15.796960 2268 raft_consensus.cc:2889] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:15.797120 2268 raft_consensus.cc:2272] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.797973 2268 ts_tablet_manager.cc:1916] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.800258 772 raft_consensus.cc:2468] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:15.801076 1176 tablet_bootstrap.cc:492] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:15.801273 1176 ts_tablet_manager.cc:1403] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.101s user 0.004s sys 0.000s
I20260505 07:54:15.760569 2276 raft_consensus.cc:2243] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.802815 2276 raft_consensus.cc:2272] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.804247 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:15.804849 2276 ts_tablet_manager.cc:1916] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.805253 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
W20260505 07:54:15.805475 856 consensus_peers.cc:597] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c1a61c1d03d544c8aa01537ead97f83c. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:15.805833 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:15.806061 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:15.806188 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:15.806290 1348 catalog_manager.cc:5671] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:15.807665 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:15.808262 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:15.809002 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:15.809649 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:15.810103 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:15.810523 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:15.810957 1348 catalog_manager.cc:5671] T 4e99f96a753047348371bee5c022684e P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 1 to 2, leader changed from d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3) to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 2 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:15.811853 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:15.817380 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:15.818111 1176 raft_consensus.cc:359] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.818267 1176 raft_consensus.cc:385] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.818315 1176 raft_consensus.cc:740] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.818545 1176 consensus_queue.cc:260] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.818202 1348 catalog_manager.cc:5671] T 3f869abaa31e414ea0625e475db08e64 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:15.819142 1176 ts_tablet_manager.cc:1434] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260505 07:54:15.819377 1176 tablet_bootstrap.cc:492] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:15.819617 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:15.820494 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:15.821484 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:15.821554 2268 ts_tablet_manager.cc:1929] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.822257 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
W20260505 07:54:15.821810 991 consensus_peers.cc:597] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 78a6470720984761a8606abeafbef992. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:15.822698 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:15.822885 2268 log.cc:1199] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/02716f62aa354d8f839266661b7f05ea
I20260505 07:54:15.823101 2276 ts_tablet_manager.cc:1929] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.823537 2268 ts_tablet_manager.cc:1950] T 02716f62aa354d8f839266661b7f05ea P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.823764 2276 log.cc:1199] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/df1b62a30b2842b897a04c793bb5e7ce
I20260505 07:54:15.824216 2276 ts_tablet_manager.cc:1950] T df1b62a30b2842b897a04c793bb5e7ce P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.829073 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:15.830874 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:15.831434 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:15.832531 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:15.833002 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:15.833616 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 158) took 5985 ms. Trace:
I20260505 07:54:15.833643 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:15.833844 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:15.834071 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:15.834934 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 125) took 8051 ms. Trace:
I20260505 07:54:15.835049 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:15.834466 1181 tablet_bootstrap.cc:492] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.835300 1181 ts_tablet_manager.cc:1403] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.046s user 0.003s sys 0.003s
I20260505 07:54:15.835304 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:15.836133 1181 raft_consensus.cc:359] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.836390 1181 raft_consensus.cc:385] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.836495 1181 raft_consensus.cc:740] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.836825 1181 consensus_queue.cc:260] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.837469 1181 ts_tablet_manager.cc:1434] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:15.837811 1181 tablet_bootstrap.cc:492] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.839361 1181 tablet_bootstrap.cc:654] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.839632 2276 rpcz_store.cc:276] 0505 07:54:07.783250 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.801311 (+ 18061us) service_pool.cc:224] Handling call
0505 07:54:15.760341 (+7959030us) ts_tablet_manager.cc:1151] Deleting tablet df1b62a30b2842b897a04c793bb5e7ce
0505 07:54:15.760350 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.808440 (+ 48090us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.811537 (+ 3097us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.834915 (+ 23378us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7958921}
I20260505 07:54:15.840016 2276 tablet_replica.cc:333] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.840170 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet df1b62a30b2842b897a04c793bb5e7ce (table default.Table_6bd554849539457bb38dbbef32deb609 [id=269f2d2935a644518501cb0700a7cff9]) successfully deleted
I20260505 07:54:15.840216 2276 raft_consensus.cc:2243] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.840461 2276 raft_consensus.cc:2272] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.840924 2276 ts_tablet_manager.cc:1916] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.835090 2268 rpcz_store.cc:276] 0505 07:54:09.847762 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.859370 (+ 11608us) service_pool.cc:224] Handling call
0505 07:54:15.796077 (+5936707us) ts_tablet_manager.cc:1151] Deleting tablet 02716f62aa354d8f839266661b7f05ea
0505 07:54:15.796090 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.800872 (+ 4782us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.821510 (+ 20638us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.833597 (+ 12087us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":397,"tablet-delete.queue_time_us":5936518}
I20260505 07:54:15.843851 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:15.844262 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 159) took 5983 ms. Trace:
I20260505 07:54:15.844384 2268 rpcz_store.cc:276] 0505 07:54:09.860414 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.861514 (+ 1100us) service_pool.cc:224] Handling call
0505 07:54:15.844112 (+5982598us) ts_tablet_manager.cc:1151] Deleting tablet 05351965bdba4ccca8d13223cd505948
0505 07:54:15.844125 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.844240 (+ 115us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5982427}
I20260505 07:54:15.844720 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
W20260505 07:54:15.845400 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 05351965bdba4ccca8d13223cd505948 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05351965bdba4ccca8d13223cd505948 already in progress: creating tablet
I20260505 07:54:15.845701 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:15.846202 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:15.846799 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 160) took 5984 ms. Trace:
I20260505 07:54:15.846884 2268 rpcz_store.cc:276] 0505 07:54:09.861902 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.863749 (+ 1847us) service_pool.cc:224] Handling call
0505 07:54:15.846721 (+5982972us) ts_tablet_manager.cc:1151] Deleting tablet 976359601ad74b149696dc7f555b550c
0505 07:54:15.846730 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.846784 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5982783}
I20260505 07:54:15.847486 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
W20260505 07:54:15.847862 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 976359601ad74b149696dc7f555b550c because the tablet was not found. No further retry: Not found: Tablet not found: 976359601ad74b149696dc7f555b550c
I20260505 07:54:15.848165 2268 tablet_replica.cc:333] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.848301 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:15.848894 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:15.849331 2268 raft_consensus.cc:2243] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.849436 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:15.849539 2268 raft_consensus.cc:2272] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.850163 2268 ts_tablet_manager.cc:1916] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.852404 1050 raft_consensus.cc:3060] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.854357 3028 raft_consensus.cc:493] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.854457 3028 raft_consensus.cc:515] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.855011 3028 leader_election.cc:290] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.855468 2872 consensus_queue.cc:1048] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.855747 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3be796a788e4465a36bb5be44918be1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.855820 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:15.855912 921 raft_consensus.cc:2468] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:15.856256 991 leader_election.cc:304] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:15.856621 2650 raft_consensus.cc:2804] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:15.856616 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:15.856686 2650 raft_consensus.cc:493] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:15.856735 2650 raft_consensus.cc:3060] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.857303 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:15.857529 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:15.857667 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:15.857985 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:15.858237 1348 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:15.859006 2650 raft_consensus.cc:515] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.859498 2650 leader_election.cc:290] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.860298 1181 tablet_bootstrap.cc:492] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.860459 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3be796a788e4465a36bb5be44918be1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:15.860505 1181 ts_tablet_manager.cc:1403] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.023s user 0.000s sys 0.003s
I20260505 07:54:15.860599 921 raft_consensus.cc:3060] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.861409 1181 raft_consensus.cc:359] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.861577 1181 raft_consensus.cc:385] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.861631 1181 raft_consensus.cc:740] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.861889 1181 consensus_queue.cc:260] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.862524 1181 ts_tablet_manager.cc:1434] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:15.862757 1181 tablet_bootstrap.cc:492] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.863461 3028 raft_consensus.cc:493] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.863554 3028 raft_consensus.cc:515] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.864013 3028 leader_election.cc:290] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.864203 921 raft_consensus.cc:2468] T c3be796a788e4465a36bb5be44918be1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:15.864650 991 leader_election.cc:304] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:15.864814 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b528d6c36ac4920b6ea0a6e26cca269" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.864945 3028 raft_consensus.cc:2804] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:15.864981 907 raft_consensus.cc:2468] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:15.865051 3028 raft_consensus.cc:697] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:15.865316 991 leader_election.cc:304] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:15.865257 3028 consensus_queue.cc:237] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.865583 2948 raft_consensus.cc:2804] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:15.865644 2948 raft_consensus.cc:493] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:15.865702 2948 raft_consensus.cc:3060] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.868614 2948 raft_consensus.cc:515] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.869266 2948 leader_election.cc:290] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:15.869535 420 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu pid 1818
I20260505 07:54:15.871121 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b528d6c36ac4920b6ea0a6e26cca269" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:15.871268 921 raft_consensus.cc:3060] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.871189 1181 tablet_bootstrap.cc:654] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.874249 921 raft_consensus.cc:2468] T 4b528d6c36ac4920b6ea0a6e26cca269 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:15.874639 991 leader_election.cc:304] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:15.874897 2948 raft_consensus.cc:2804] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:15.875039 2948 raft_consensus.cc:697] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:15.875281 2948 consensus_queue.cc:237] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.877156 2268 ts_tablet_manager.cc:1929] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.877566 2276 ts_tablet_manager.cc:1929] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.878584 2276 log.cc:1199] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/ba616702dc6040bb805bb3ffc8a8a871
W20260505 07:54:15.882319 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 131089us
I20260505 07:54:15.888867 2447 log.cc:1199] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/323269e708964848a94c64e6a781a860
I20260505 07:54:15.889259 2447 ts_tablet_manager.cc:1950] T 323269e708964848a94c64e6a781a860 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:15.889884 2276 ts_tablet_manager.cc:1950] T ba616702dc6040bb805bb3ffc8a8a871 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.890138 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 173) took 6338 ms. Trace:
I20260505 07:54:15.890221 2447 rpcz_store.cc:276] 0505 07:54:09.551597 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.551691 (+ 94us) service_pool.cc:224] Handling call
0505 07:54:15.652883 (+6101192us) ts_tablet_manager.cc:1151] Deleting tablet 323269e708964848a94c64e6a781a860
0505 07:54:15.652891 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.724120 (+ 71229us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.746447 (+ 22327us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.890120 (+143673us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6093957}
I20260505 07:54:15.890599 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 323269e708964848a94c64e6a781a860 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]) successfully deleted
I20260505 07:54:15.890682 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 174) took 6338 ms. Trace:
I20260505 07:54:15.890748 2447 rpcz_store.cc:276] 0505 07:54:09.552073 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.552122 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:15.890587 (+6338465us) ts_tablet_manager.cc:1151] Deleting tablet e278b86e7b5d444ebad8d0f6921d4c62
0505 07:54:15.890597 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.890668 (+ 71us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6331617}
I20260505 07:54:15.891101 2447 tablet_replica.cc:333] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
W20260505 07:54:15.891116 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet e278b86e7b5d444ebad8d0f6921d4c62 because the tablet was not found. No further retry: Not found: Tablet not found: e278b86e7b5d444ebad8d0f6921d4c62
I20260505 07:54:15.891271 2447 raft_consensus.cc:2243] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.891398 2447 raft_consensus.cc:2272] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.891885 2447 ts_tablet_manager.cc:1916] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.888278 1878 consensus_queue.cc:1048] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.892071 2268 log.cc:1199] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/1ef4f516e64a44fd8cd33674fb808067
I20260505 07:54:15.892891 2268 ts_tablet_manager.cc:1950] T 1ef4f516e64a44fd8cd33674fb808067 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.893110 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 126) took 8109 ms. Trace:
I20260505 07:54:15.893196 2276 rpcz_store.cc:276] 0505 07:54:07.783328 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.801431 (+ 18103us) service_pool.cc:224] Handling call
0505 07:54:15.839960 (+8038529us) ts_tablet_manager.cc:1151] Deleting tablet ba616702dc6040bb805bb3ffc8a8a871
0505 07:54:15.839970 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.852953 (+ 12983us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.877464 (+ 24511us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.893085 (+ 15621us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8038389}
I20260505 07:54:15.893682 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 161) took 6029 ms. Trace:
I20260505 07:54:15.894258 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet ba616702dc6040bb805bb3ffc8a8a871 (table default.Table_2717154d3561495692818d7755d53c41 [id=23c6264859f64b598af609ed2fff6e8f]) successfully deleted
I20260505 07:54:15.894608 2276 tablet_replica.cc:333] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.893767 2268 rpcz_store.cc:276] 0505 07:54:09.864148 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.864508 (+ 360us) service_pool.cc:224] Handling call
0505 07:54:15.848123 (+5983615us) ts_tablet_manager.cc:1151] Deleting tablet 1ef4f516e64a44fd8cd33674fb808067
0505 07:54:15.848134 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.857903 (+ 9769us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.877104 (+ 19201us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.893652 (+ 16548us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5983350}
I20260505 07:54:15.895236 2276 raft_consensus.cc:2243] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.895635 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 1ef4f516e64a44fd8cd33674fb808067 (table default.Table_b85a2b107c88498e84217252ca6cb947 [id=2c91868ddba84103b9a5c78201842a9e]) successfully deleted
I20260505 07:54:15.896332 2276 raft_consensus.cc:2272] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.895977 2268 tablet_replica.cc:333] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.896586 2268 raft_consensus.cc:2243] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.897213 2276 ts_tablet_manager.cc:1916] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.897452 2268 raft_consensus.cc:2272] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.898561 1181 tablet_bootstrap.cc:492] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.899361 1181 ts_tablet_manager.cc:1403] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.037s user 0.002s sys 0.002s
I20260505 07:54:15.900279 1181 raft_consensus.cc:359] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.900512 1181 raft_consensus.cc:385] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.900610 1181 raft_consensus.cc:740] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.900882 1181 consensus_queue.cc:260] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.901387 1181 ts_tablet_manager.cc:1434] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:15.901654 1181 tablet_bootstrap.cc:492] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.913468 2268 ts_tablet_manager.cc:1916] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.915199 1181 tablet_bootstrap.cc:654] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.918533 1172 tablet_bootstrap.cc:492] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:15.918704 1172 ts_tablet_manager.cc:1403] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.143s user 0.007s sys 0.023s
I20260505 07:54:15.920056 1172 raft_consensus.cc:359] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.920215 1172 raft_consensus.cc:385] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.920279 1172 raft_consensus.cc:740] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.920447 1172 consensus_queue.cc:260] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.920920 1172 ts_tablet_manager.cc:1434] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:15.921166 1172 tablet_bootstrap.cc:492] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:15.922530 1172 tablet_bootstrap.cc:654] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.928109 2276 ts_tablet_manager.cc:1929] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.929558 2268 ts_tablet_manager.cc:1929] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.929970 2276 log.cc:1199] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/d7e0c8ad716e443588d4ae3aa28ed4bf
I20260505 07:54:15.929396 2752 raft_consensus.cc:493] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.929504 3045 consensus_queue.cc:1048] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.930251 2752 raft_consensus.cc:515] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.929695 2505 raft_consensus.cc:493] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.930608 2505 raft_consensus.cc:515] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.930770 2752 leader_election.cc:290] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.931100 2505 leader_election.cc:290] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.928627 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9963a0f1a54ee9a73486bb1801ee74" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:15.931263 918 raft_consensus.cc:3060] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:15.933883 918 raft_consensus.cc:2468] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:15.934974 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b9963a0f1a54ee9a73486bb1801ee74" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:15.935108 907 raft_consensus.cc:2376] T 4b9963a0f1a54ee9a73486bb1801ee74 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1. Re-sending same reply.
I20260505 07:54:15.935315 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e608ac01ef4a88906e585bd7ee25ac" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.935694 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310ef5aef7e44ec5a062a01d2a175a11" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.935964 751 tablet_service.cc:1511] Processing CreateTablet for tablet 0a2e22cb61c340e289afd6cffbfc7661 (DEFAULT_TABLE table=default.Table_fea44fc5aa9a4967a7e0ca87a0954aab [id=ffafde3a11014053ae32c590c0c92aad]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:15.936072 854 leader_election.cc:343] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:15.936323 854 leader_election.cc:343] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.936913 744 tablet_service.cc:1511] Processing CreateTablet for tablet 1456eecef5794452b8446596be57156e (DEFAULT_TABLE table=default.Table_8f7522cabddb40e0b938edb0f45626c3 [id=fb8f0aca7a7140b9a42800dfe80b405f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:15.936962 749 tablet_service.cc:1511] Processing CreateTablet for tablet c8c61afc74ec47fbac88d7237ce7d688 (DEFAULT_TABLE table=default.Table_df175162fead408e9350bc9eb1de361c [id=03c1adbec92d4790a3c0f979bf5bf2a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:15.938259 885 tablet_service.cc:1511] Processing CreateTablet for tablet 0a2e22cb61c340e289afd6cffbfc7661 (DEFAULT_TABLE table=default.Table_fea44fc5aa9a4967a7e0ca87a0954aab [id=ffafde3a11014053ae32c590c0c92aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:15.938299 896 tablet_service.cc:1511] Processing CreateTablet for tablet c8c61afc74ec47fbac88d7237ce7d688 (DEFAULT_TABLE table=default.Table_df175162fead408e9350bc9eb1de361c [id=03c1adbec92d4790a3c0f979bf5bf2a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:15.938563 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a2e22cb61c340e289afd6cffbfc7661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:15.938750 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c61afc74ec47fbac88d7237ce7d688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:15.938997 899 tablet_service.cc:1511] Processing CreateTablet for tablet 1456eecef5794452b8446596be57156e (DEFAULT_TABLE table=default.Table_8f7522cabddb40e0b938edb0f45626c3 [id=fb8f0aca7a7140b9a42800dfe80b405f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:15.939281 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1456eecef5794452b8446596be57156e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:15.951520 2752 consensus_queue.cc:1048] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:15.959858 2276 ts_tablet_manager.cc:1950] T d7e0c8ad716e443588d4ae3aa28ed4bf P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.960188 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 127) took 8176 ms. Trace:
I20260505 07:54:15.960295 2276 rpcz_store.cc:276] 0505 07:54:07.783408 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.801581 (+ 18173us) service_pool.cc:224] Handling call
0505 07:54:15.894561 (+8092980us) ts_tablet_manager.cc:1151] Deleting tablet d7e0c8ad716e443588d4ae3aa28ed4bf
0505 07:54:15.894572 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.920822 (+ 26250us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.928065 (+ 7243us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.960176 (+ 32111us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8092868}
I20260505 07:54:15.960649 2276 tablet_replica.cc:333] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.960790 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet d7e0c8ad716e443588d4ae3aa28ed4bf (table default.Table_314825f5a7f644dba828c34f3d2d740a [id=056371ab05fc41c78f31a8ac6887ecb5]) successfully deleted
I20260505 07:54:15.960845 2276 raft_consensus.cc:2243] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:15.960976 2276 raft_consensus.cc:2272] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.960983 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed0d43d85fcd4d05abf10d415e8dafbe" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:15.961153 2268 log.cc:1199] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/7e707143a4e64725aa66db05b24999a6
I20260505 07:54:15.931790 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 0a2e22cb61c340e289afd6cffbfc7661 (DEFAULT_TABLE table=default.Table_fea44fc5aa9a4967a7e0ca87a0954aab [id=ffafde3a11014053ae32c590c0c92aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:15.961472 2268 ts_tablet_manager.cc:1950] T 7e707143a4e64725aa66db05b24999a6 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.961555 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a2e22cb61c340e289afd6cffbfc7661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:15.961740 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 162) took 6096 ms. Trace:
I20260505 07:54:15.961794 2268 rpcz_store.cc:276] 0505 07:54:09.864904 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.867094 (+ 2190us) service_pool.cc:224] Handling call
0505 07:54:15.895927 (+6028833us) ts_tablet_manager.cc:1151] Deleting tablet 7e707143a4e64725aa66db05b24999a6
0505 07:54:15.895938 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.922948 (+ 27010us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.929502 (+ 6554us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.961727 (+ 32225us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6028660}
I20260505 07:54:15.962103 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 7e707143a4e64725aa66db05b24999a6 (table default.Table_f0316c88590245aea8815b3b08e89f45 [id=c3a480b747de444aa45b771244908352]) successfully deleted
I20260505 07:54:15.962839 2276 ts_tablet_manager.cc:1916] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.937389 1020 tablet_service.cc:1511] Processing CreateTablet for tablet 1456eecef5794452b8446596be57156e (DEFAULT_TABLE table=default.Table_8f7522cabddb40e0b938edb0f45626c3 [id=fb8f0aca7a7140b9a42800dfe80b405f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:15.963822 1020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1456eecef5794452b8446596be57156e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:15.964762 2268 tablet_replica.cc:333] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.964938 2268 raft_consensus.cc:2243] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.965202 2268 raft_consensus.cc:2272] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.965636 2268 ts_tablet_manager.cc:1916] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.937366 1023 tablet_service.cc:1511] Processing CreateTablet for tablet c8c61afc74ec47fbac88d7237ce7d688 (DEFAULT_TABLE table=default.Table_df175162fead408e9350bc9eb1de361c [id=03c1adbec92d4790a3c0f979bf5bf2a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:15.958124 1181 tablet_bootstrap.cc:492] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.967746 1181 ts_tablet_manager.cc:1403] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.066s user 0.003s sys 0.001s
I20260505 07:54:15.967828 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c61afc74ec47fbac88d7237ce7d688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:15.968493 1181 raft_consensus.cc:359] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.968621 1181 raft_consensus.cc:385] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.968665 1181 raft_consensus.cc:740] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.968848 1181 consensus_queue.cc:260] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.969286 1181 ts_tablet_manager.cc:1434] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260505 07:54:15.969568 1181 tablet_bootstrap.cc:492] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:15.969987 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:15.970243 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:15.970541 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:15.970783 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:15.970953 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:15.971160 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:15.971204 1181 tablet_bootstrap.cc:654] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.971307 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:15.971524 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:15.971808 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:15.971997 2856 raft_consensus.cc:493] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.972118 2856 raft_consensus.cc:515] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.972203 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:15.959211 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310ef5aef7e44ec5a062a01d2a175a11" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.972468 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:15.972661 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:15.972813 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:15.972849 2856 leader_election.cc:290] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.972995 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:15.973162 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
W20260505 07:54:15.972827 856 leader_election.cc:343] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.973500 856 leader_election.cc:304] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.973703 2856 raft_consensus.cc:2749] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.973862 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:15.974278 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36a0f68fc2b4adda04ec3b9db5f2d8e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:15.974669 856 leader_election.cc:343] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.959376 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e608ac01ef4a88906e585bd7ee25ac" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:15.975914 856 leader_election.cc:343] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.976002 856 leader_election.cc:304] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.976181 2856 raft_consensus.cc:2749] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.976293 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36a0f68fc2b4adda04ec3b9db5f2d8e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:15.976841 854 leader_election.cc:343] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.976940 854 leader_election.cc:304] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.977288 2856 raft_consensus.cc:2749] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.978546 2856 raft_consensus.cc:493] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:15.978672 2856 raft_consensus.cc:515] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.979210 2856 leader_election.cc:290] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:15.979650 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1d3829ede4b5582a022b64265df7e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:15.979693 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1d3829ede4b5582a022b64265df7e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:15.980069 854 leader_election.cc:343] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:15.980182 856 leader_election.cc:343] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:15.980281 856 leader_election.cc:304] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:15.980458 2856 raft_consensus.cc:2749] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:15.985392 2268 ts_tablet_manager.cc:1929] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.985487 2268 log.cc:1199] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/1852f7f4ff594639908b03730b68fe86
I20260505 07:54:15.985900 2268 ts_tablet_manager.cc:1950] T 1852f7f4ff594639908b03730b68fe86 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:15.986217 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 163) took 6118 ms. Trace:
I20260505 07:54:15.986305 2268 rpcz_store.cc:276] 0505 07:54:09.867494 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.869427 (+ 1933us) service_pool.cc:224] Handling call
0505 07:54:15.961988 (+6092561us) ts_tablet_manager.cc:1151] Deleting tablet 1852f7f4ff594639908b03730b68fe86
0505 07:54:15.964728 (+ 2740us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.968294 (+ 3566us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.985353 (+ 17059us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.986199 (+ 846us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6092428}
I20260505 07:54:15.986807 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 1852f7f4ff594639908b03730b68fe86 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]) successfully deleted
I20260505 07:54:15.987504 2276 ts_tablet_manager.cc:1929] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:15.987957 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:15.988111 2276 log.cc:1199] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/080b2984fcf2478a8def6540da63554d
I20260505 07:54:15.988451 2276 ts_tablet_manager.cc:1950] T 080b2984fcf2478a8def6540da63554d P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:15.988775 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:15.989194 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 128) took 8205 ms. Trace:
I20260505 07:54:15.989540 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:15.989718 2276 rpcz_store.cc:276] 0505 07:54:07.783478 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.801700 (+ 18222us) service_pool.cc:224] Handling call
0505 07:54:15.960593 (+8158893us) ts_tablet_manager.cc:1151] Deleting tablet 080b2984fcf2478a8def6540da63554d
0505 07:54:15.960601 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:15.975021 (+ 14420us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.987465 (+ 12444us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:15.989181 (+ 1716us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8158783}
I20260505 07:54:15.990521 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:15.990825 2276 tablet_replica.cc:333] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:15.991106 2276 raft_consensus.cc:2243] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.991511 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 080b2984fcf2478a8def6540da63554d (table default.Table_575ab5b128ee4a1baebbe1f90baf005a [id=cdb574c7a4e14a5ea7f3c8b036a8c336]) successfully deleted
I20260505 07:54:15.992034 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:15.992472 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:15.991532 2276 raft_consensus.cc:2272] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.993379 2276 ts_tablet_manager.cc:1916] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.994534 2268 tablet_replica.cc:333] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:15.994721 2268 raft_consensus.cc:2243] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:15.994951 2268 raft_consensus.cc:2272] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:15.995393 2268 ts_tablet_manager.cc:1916] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:15.996037 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:15.996358 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:15.996691 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:15.996857 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:15.997450 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:15.997609 1176 tablet_bootstrap.cc:654] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:15.997691 1181 tablet_bootstrap.cc:492] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:15.997812 1181 ts_tablet_manager.cc:1403] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.007s
I20260505 07:54:15.997862 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:15.998507 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:15.998525 1181 raft_consensus.cc:359] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.998679 1181 raft_consensus.cc:385] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:15.998739 1181 raft_consensus.cc:740] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:15.998936 1181 consensus_queue.cc:260] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:15.999498 1181 ts_tablet_manager.cc:1434] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:15.999759 1181 tablet_bootstrap.cc:492] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.001271 1181 tablet_bootstrap.cc:654] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.001307 1172 tablet_bootstrap.cc:492] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:16.001499 1172 ts_tablet_manager.cc:1403] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.080s user 0.001s sys 0.001s
I20260505 07:54:16.002226 1172 raft_consensus.cc:359] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.006225 1172 raft_consensus.cc:385] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.006664 1172 raft_consensus.cc:740] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.007017 1172 consensus_queue.cc:260] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.007970 1172 ts_tablet_manager.cc:1434] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260505 07:54:16.007994 1181 tablet_bootstrap.cc:492] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.008090 1181 ts_tablet_manager.cc:1403] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260505 07:54:16.008394 1172 tablet_bootstrap.cc:492] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:16.008720 1181 raft_consensus.cc:359] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.008896 1181 raft_consensus.cc:385] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.008960 1181 raft_consensus.cc:740] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.009146 1181 consensus_queue.cc:260] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.009626 1181 ts_tablet_manager.cc:1434] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260505 07:54:16.009884 1181 tablet_bootstrap.cc:492] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.009954 2268 ts_tablet_manager.cc:1929] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.010042 2268 log.cc:1199] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/397d33e908af4a0bb8dde26d273be01a
I20260505 07:54:16.010344 2268 ts_tablet_manager.cc:1950] T 397d33e908af4a0bb8dde26d273be01a P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.010350 1172 tablet_bootstrap.cc:654] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.010651 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 164) took 6140 ms. Trace:
I20260505 07:54:16.010746 2268 rpcz_store.cc:276] 0505 07:54:09.869728 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.870696 (+ 968us) service_pool.cc:224] Handling call
0505 07:54:15.994495 (+6123799us) ts_tablet_manager.cc:1151] Deleting tablet 397d33e908af4a0bb8dde26d273be01a
0505 07:54:15.994504 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.003107 (+ 8603us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.009914 (+ 6807us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.010620 (+ 706us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6123383}
I20260505 07:54:16.011168 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 397d33e908af4a0bb8dde26d273be01a (table default.Table_499d356daab040e29ca2f5eb2941d790 [id=bf9eb924eb1e4586a6346bcf43be9a30]) successfully deleted
I20260505 07:54:16.011440 1181 tablet_bootstrap.cc:654] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.011466 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:16.011943 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:16.012403 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:16.012650 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:16.013250 2276 ts_tablet_manager.cc:1929] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.013757 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:16.013928 2276 log.cc:1199] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/472441bd29704be59aabb0f7a4964bb0
I20260505 07:54:16.014279 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:16.014673 2276 ts_tablet_manager.cc:1950] T 472441bd29704be59aabb0f7a4964bb0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.014966 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 129) took 8231 ms. Trace:
I20260505 07:54:16.015621 2276 rpcz_store.cc:276] 0505 07:54:07.783530 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.801834 (+ 18304us) service_pool.cc:224] Handling call
0505 07:54:15.990785 (+8188951us) ts_tablet_manager.cc:1151] Deleting tablet 472441bd29704be59aabb0f7a4964bb0
0505 07:54:15.990794 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.005072 (+ 14278us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.013212 (+ 8140us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.014952 (+ 1740us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8188837}
I20260505 07:54:16.016570 2276 tablet_replica.cc:333] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.016759 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:16.016988 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 472441bd29704be59aabb0f7a4964bb0 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]) successfully deleted
I20260505 07:54:16.017222 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:16.017861 1181 tablet_bootstrap.cc:492] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.017907 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:16.018006 1181 ts_tablet_manager.cc:1403] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260505 07:54:16.018126 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:16.018337 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:16.018688 1181 raft_consensus.cc:359] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.018796 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:16.018846 1181 raft_consensus.cc:385] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.018918 1181 raft_consensus.cc:740] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.019124 2650 raft_consensus.cc:493] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:16.019212 1355 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:16.019235 2650 raft_consensus.cc:515] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.019762 2650 leader_election.cc:290] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:16.020852 991 leader_election.cc:343] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: a9494c3b6a99487596a030856b371b68
I20260505 07:54:16.021098 1181 consensus_queue.cc:260] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.021657 1181 ts_tablet_manager.cc:1434] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260505 07:54:16.021996 1181 tablet_bootstrap.cc:492] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.023442 1181 tablet_bootstrap.cc:654] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.024294 2276 raft_consensus.cc:2243] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.024448 2276 raft_consensus.cc:2272] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.024927 2276 ts_tablet_manager.cc:1916] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.025015 1172 tablet_bootstrap.cc:492] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:16.025133 1172 ts_tablet_manager.cc:1403] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.004s
I20260505 07:54:16.025185 2268 tablet_replica.cc:333] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.025365 2268 raft_consensus.cc:2243] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.025497 2268 raft_consensus.cc:2272] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.025731 1172 raft_consensus.cc:359] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.025882 1172 raft_consensus.cc:385] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.025943 1172 raft_consensus.cc:740] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.026125 1172 consensus_queue.cc:260] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.026710 1172 ts_tablet_manager.cc:1434] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:16.026957 1172 tablet_bootstrap.cc:492] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:16.028154 2268 ts_tablet_manager.cc:1916] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.028515 1172 tablet_bootstrap.cc:654] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.029997 2856 raft_consensus.cc:493] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.030114 2856 raft_consensus.cc:515] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.030611 2856 leader_election.cc:290] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.031028 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b46ff0101a4e529077b9e30492ace7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.031239 1052 raft_consensus.cc:2468] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:16.031626 856 leader_election.cc:304] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:16.031843 2856 raft_consensus.cc:2804] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:16.031924 2856 raft_consensus.cc:493] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:16.031999 2856 raft_consensus.cc:3060] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.034318 2856 raft_consensus.cc:515] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.039068 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b46ff0101a4e529077b9e30492ace7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:16.039261 777 raft_consensus.cc:3060] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.039801 2856 leader_election.cc:290] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.040284 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b46ff0101a4e529077b9e30492ace7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:16.040453 1052 raft_consensus.cc:3060] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.041946 777 raft_consensus.cc:2468] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:16.042433 854 leader_election.cc:304] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:16.042639 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09b46ff0101a4e529077b9e30492ace7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.042824 775 raft_consensus.cc:2376] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1. Re-sending same reply.
I20260505 07:54:16.043720 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3be796a788e4465a36bb5be44918be1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:16.043917 777 raft_consensus.cc:3060] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.045490 1052 raft_consensus.cc:2468] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:16.046482 777 raft_consensus.cc:2468] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:16.046746 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3be796a788e4465a36bb5be44918be1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.046953 775 raft_consensus.cc:2376] T c3be796a788e4465a36bb5be44918be1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1. Re-sending same reply.
I20260505 07:54:16.048103 1172 tablet_bootstrap.cc:492] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:16.048242 1172 ts_tablet_manager.cc:1403] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.000s
I20260505 07:54:16.049394 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b528d6c36ac4920b6ea0a6e26cca269" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.049477 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b528d6c36ac4920b6ea0a6e26cca269" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:16.049598 775 raft_consensus.cc:2468] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.051746 1176 tablet_bootstrap.cc:492] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:16.051903 1176 ts_tablet_manager.cc:1403] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.233s user 0.004s sys 0.000s
I20260505 07:54:16.052547 1176 raft_consensus.cc:359] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.052686 1176 raft_consensus.cc:385] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.052762 1176 raft_consensus.cc:740] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.052985 1176 consensus_queue.cc:260] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.053498 1176 ts_tablet_manager.cc:1434] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:16.053761 1176 tablet_bootstrap.cc:492] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:16.054904 1176 tablet_bootstrap.cc:654] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.058666 2268 ts_tablet_manager.cc:1929] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:16.058784 2268 log.cc:1199] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/ff7ed926a3ff4e7993864de734ca3c50
I20260505 07:54:16.059084 2268 ts_tablet_manager.cc:1950] T ff7ed926a3ff4e7993864de734ca3c50 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.059350 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 165) took 6188 ms. Trace:
I20260505 07:54:16.059427 2268 rpcz_store.cc:276] 0505 07:54:09.870903 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.871303 (+ 400us) service_pool.cc:224] Handling call
0505 07:54:16.025145 (+6153842us) ts_tablet_manager.cc:1151] Deleting tablet ff7ed926a3ff4e7993864de734ca3c50
0505 07:54:16.025154 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.055950 (+ 30796us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.058614 (+ 2664us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.059328 (+ 714us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6153688}
I20260505 07:54:16.059917 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet ff7ed926a3ff4e7993864de734ca3c50 (table default.Table_1b644b185b194940a8a8a90595ae9e90 [id=8e18fbfbd36340fabddf08dc99fea437]) successfully deleted
I20260505 07:54:16.060079 1181 tablet_bootstrap.cc:492] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.060189 1181 ts_tablet_manager.cc:1403] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.038s user 0.004s sys 0.004s
I20260505 07:54:16.060990 1181 raft_consensus.cc:359] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.061167 1181 raft_consensus.cc:385] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.061220 1181 raft_consensus.cc:740] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.061391 1181 consensus_queue.cc:260] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.061813 1181 ts_tablet_manager.cc:1434] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:16.062048 1181 tablet_bootstrap.cc:492] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.062727 2276 ts_tablet_manager.cc:1929] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.063766 1181 tablet_bootstrap.cc:654] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.063903 2276 log.cc:1199] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/aab3850799254dda9716fd5bf9404159
I20260505 07:54:16.064297 2276 ts_tablet_manager.cc:1950] T aab3850799254dda9716fd5bf9404159 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.067214 2856 raft_consensus.cc:2804] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:16.067328 3045 raft_consensus.cc:493] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.068156 3045 raft_consensus.cc:515] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.068755 3045 leader_election.cc:290] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.067637 2752 raft_consensus.cc:493] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.068984 2752 raft_consensus.cc:515] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.069479 2752 leader_election.cc:290] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.069692 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7071df211bd34b86a751142bae3271b2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:16.070937 856 leader_election.cc:343] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.071224 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172ca66c49354f549fa55fe63abf9970" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:16.071784 856 leader_election.cc:343] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.073572 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172ca66c49354f549fa55fe63abf9970" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.073968 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7071df211bd34b86a751142bae3271b2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.075006 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 130) took 8291 ms. Trace:
I20260505 07:54:16.075088 2276 rpcz_store.cc:276] 0505 07:54:07.783609 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.801955 (+ 18346us) service_pool.cc:224] Handling call
0505 07:54:16.016007 (+8214052us) ts_tablet_manager.cc:1151] Deleting tablet aab3850799254dda9716fd5bf9404159
0505 07:54:16.016507 (+ 500us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.037249 (+ 20742us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.062674 (+ 25425us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.074989 (+ 12315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816,"tablet-delete.queue_time_us":8213924}
I20260505 07:54:16.075379 2276 tablet_replica.cc:333] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.076225 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet aab3850799254dda9716fd5bf9404159 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]) successfully deleted
W20260505 07:54:16.075552 854 leader_election.cc:343] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.076911 854 leader_election.cc:304] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.077101 2752 raft_consensus.cc:2749] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:16.077256 854 leader_election.cc:343] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.077334 854 leader_election.cc:304] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.077479 2752 raft_consensus.cc:2749] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:16.078001 856 consensus_peers.cc:597] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 93d878ae0ddb42319560f74e0a490321. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:16.078532 2268 tablet_replica.cc:333] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.079298 2268 raft_consensus.cc:2243] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.079712 2268 raft_consensus.cc:2272] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.080659 2276 raft_consensus.cc:2243] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.081254 2276 raft_consensus.cc:2272] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.081929 2276 ts_tablet_manager.cc:1916] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.082332 2268 ts_tablet_manager.cc:1916] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.089664 2856 raft_consensus.cc:697] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:16.090034 2856 consensus_queue.cc:237] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.092243 3078 raft_consensus.cc:493] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.092394 3078 raft_consensus.cc:515] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.093132 3078 leader_election.cc:290] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.093360 1878 raft_consensus.cc:493] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.093317 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49d7b9f5b6f74f8ea070f8ea7e545df7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.093477 1878 raft_consensus.cc:515] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:16.093760 854 leader_election.cc:343] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.094043 1878 leader_election.cc:290] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.094991 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2045202714c14aa388d1e8bc18e2063d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:16.095184 921 raft_consensus.cc:2468] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.095559 991 leader_election.cc:304] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:16.095741 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49d7b9f5b6f74f8ea070f8ea7e545df7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.095878 1878 raft_consensus.cc:2804] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:16.095966 1878 raft_consensus.cc:493] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:16.096017 1878 raft_consensus.cc:3060] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:16.096223 856 leader_election.cc:343] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.096311 856 leader_election.cc:304] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.098711 1878 raft_consensus.cc:515] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.099597 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2045202714c14aa388d1e8bc18e2063d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:16.099735 907 raft_consensus.cc:3060] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.102805 907 raft_consensus.cc:2468] T 2045202714c14aa388d1e8bc18e2063d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:16.103317 991 leader_election.cc:304] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:16.103523 2650 raft_consensus.cc:2804] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:16.103636 2650 raft_consensus.cc:697] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:16.103837 2650 consensus_queue.cc:237] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.104745 1176 tablet_bootstrap.cc:492] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:16.104873 1176 ts_tablet_manager.cc:1403] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.051s user 0.005s sys 0.003s
I20260505 07:54:16.105001 1878 leader_election.cc:290] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.105350 1172 raft_consensus.cc:359] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.105495 1172 raft_consensus.cc:385] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.105551 1172 raft_consensus.cc:740] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.105655 1176 raft_consensus.cc:359] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.105856 1176 raft_consensus.cc:385] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.105921 1176 raft_consensus.cc:740] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.106158 1176 consensus_queue.cc:260] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.106288 1172 consensus_queue.cc:260] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.106626 1176 ts_tablet_manager.cc:1434] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:16.106849 2752 raft_consensus.cc:2749] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:16.106963 1176 tablet_bootstrap.cc:492] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:16.107430 1172 ts_tablet_manager.cc:1434] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.059s user 0.002s sys 0.000s
I20260505 07:54:16.108136 1172 tablet_bootstrap.cc:492] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:16.108698 1176 tablet_bootstrap.cc:654] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.110311 1172 tablet_bootstrap.cc:654] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:16.110309 1315 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0e08cf03ea0b495a94c6566ea6c8caad consensus state has no leader
I20260505 07:54:16.137316 1176 tablet_bootstrap.cc:492] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:16.137316 1181 tablet_bootstrap.cc:492] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.137463 1181 ts_tablet_manager.cc:1403] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.075s user 0.011s sys 0.006s
I20260505 07:54:16.137465 1176 ts_tablet_manager.cc:1403] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.031s user 0.005s sys 0.005s
I20260505 07:54:16.138159 1176 raft_consensus.cc:359] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.138159 1181 raft_consensus.cc:359] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.138305 1181 raft_consensus.cc:385] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.138305 1176 raft_consensus.cc:385] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.138353 1181 raft_consensus.cc:740] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.138353 1176 raft_consensus.cc:740] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.138549 1181 consensus_queue.cc:260] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.138550 1176 consensus_queue.cc:260] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.139045 1176 ts_tablet_manager.cc:1434] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:16.139194 1181 ts_tablet_manager.cc:1434] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:16.139282 1176 tablet_bootstrap.cc:492] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:16.139429 1181 tablet_bootstrap.cc:492] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.141294 1176 tablet_bootstrap.cc:654] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.144779 1176 tablet_bootstrap.cc:492] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:16.144886 1176 ts_tablet_manager.cc:1403] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260505 07:54:16.145583 1176 raft_consensus.cc:359] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.145748 1176 raft_consensus.cc:385] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.145807 1176 raft_consensus.cc:740] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.145985 1176 consensus_queue.cc:260] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.146422 1176 ts_tablet_manager.cc:1434] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:16.146656 1176 tablet_bootstrap.cc:492] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:16.148186 1176 tablet_bootstrap.cc:654] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.150200 2268 ts_tablet_manager.cc:1929] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.150292 2268 log.cc:1199] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/a9494c3b6a99487596a030856b371b68
I20260505 07:54:16.150588 2268 ts_tablet_manager.cc:1950] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.150939 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 166) took 6279 ms. Trace:
I20260505 07:54:16.151017 2268 rpcz_store.cc:276] 0505 07:54:09.871847 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.872072 (+ 225us) service_pool.cc:224] Handling call
0505 07:54:16.078491 (+6206419us) ts_tablet_manager.cc:1151] Deleting tablet a9494c3b6a99487596a030856b371b68
0505 07:54:16.078500 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.133030 (+ 54530us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.150149 (+ 17119us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.150926 (+ 777us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6206248}
I20260505 07:54:16.151325 2268 tablet_replica.cc:333] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.151414 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet a9494c3b6a99487596a030856b371b68 (table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]) successfully deleted
I20260505 07:54:16.151492 2268 raft_consensus.cc:2243] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.151562 2505 raft_consensus.cc:493] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.151635 2268 raft_consensus.cc:2272] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.151746 2505 raft_consensus.cc:515] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.152379 2505 leader_election.cc:290] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.153985 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f840d5f58161468385d5c4f116b78c3d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:16.154481 856 leader_election.cc:343] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.155659 2276 ts_tablet_manager.cc:1929] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.155755 2276 log.cc:1199] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/2d86b8b4dadf4fd2b6409a8cb70e313e
I20260505 07:54:16.156078 2276 ts_tablet_manager.cc:1950] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.156756 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 131) took 8373 ms. Trace:
I20260505 07:54:16.156927 1181 tablet_bootstrap.cc:654] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.156836 2276 rpcz_store.cc:276] 0505 07:54:07.783667 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.802076 (+ 18409us) service_pool.cc:224] Handling call
0505 07:54:16.075341 (+8273265us) ts_tablet_manager.cc:1151] Deleting tablet 2d86b8b4dadf4fd2b6409a8cb70e313e
0505 07:54:16.075349 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.144640 (+ 69291us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.155570 (+ 10930us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.156739 (+ 1169us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8273163}
I20260505 07:54:16.157187 2276 tablet_replica.cc:333] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.157348 2276 raft_consensus.cc:2243] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.157486 2276 raft_consensus.cc:2272] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.157968 2276 ts_tablet_manager.cc:1916] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.162308 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 2d86b8b4dadf4fd2b6409a8cb70e313e (table default.Table_ff8ae2444af4400eb86af8cc0fa3c146 [id=6a242917efbf4ae5a5fa82539d6f7b34]) successfully deleted
I20260505 07:54:16.165518 2268 ts_tablet_manager.cc:1916] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.171231 2268 ts_tablet_manager.cc:1929] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.171314 2268 log.cc:1199] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/4f3e076892234c1494b1a5c13cbfd2e2
I20260505 07:54:16.173202 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:16.173825 1354 catalog_manager.cc:5671] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:16.174258 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:16.174481 1181 tablet_bootstrap.cc:492] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.174675 1181 ts_tablet_manager.cc:1403] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.000s
I20260505 07:54:16.174552 1354 catalog_manager.cc:5671] T 18e909ee93a24ea9842e95357c2ba858 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:16.174906 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:16.175320 1354 catalog_manager.cc:5671] T b6607b7428044e428af9d765c9c9b3ba P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:16.175701 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:16.175742 1181 raft_consensus.cc:359] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.176011 1181 raft_consensus.cc:385] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.176108 1181 raft_consensus.cc:740] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.176159 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:16.176339 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:16.176481 1181 consensus_queue.cc:260] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.176575 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:16.176800 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:16.177021 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:16.177332 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:16.177464 1181 ts_tablet_manager.cc:1434] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:16.177584 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:16.177805 1181 tablet_bootstrap.cc:492] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.177795 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:16.177980 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:16.178118 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:16.178339 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:16.178869 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:16.179028 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:16.179356 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:16.179612 2268 ts_tablet_manager.cc:1950] T 4f3e076892234c1494b1a5c13cbfd2e2 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.180399 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 167) took 6307 ms. Trace:
I20260505 07:54:16.180492 2268 rpcz_store.cc:276] 0505 07:54:09.872511 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.875863 (+ 3352us) service_pool.cc:224] Handling call
0505 07:54:16.151285 (+6275422us) ts_tablet_manager.cc:1151] Deleting tablet 4f3e076892234c1494b1a5c13cbfd2e2
0505 07:54:16.151295 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.168485 (+ 17190us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.171193 (+ 2708us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.180383 (+ 9190us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6275258}
I20260505 07:54:16.180925 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 4f3e076892234c1494b1a5c13cbfd2e2 (table default.Table_fe563815e7df4183a9bfe81907c3c33c [id=fa1b1c24909247538d57d1a37ea9513b]) successfully deleted
I20260505 07:54:16.181280 2276 ts_tablet_manager.cc:1929] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.181376 2276 log.cc:1199] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/abe5f6f0774545b296bf1b03a08e145d
I20260505 07:54:16.181671 2276 ts_tablet_manager.cc:1950] T abe5f6f0774545b296bf1b03a08e145d P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.181994 2268 tablet_replica.cc:333] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.182185 2268 raft_consensus.cc:2243] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:16.182425 2268 raft_consensus.cc:2272] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.182905 1878 raft_consensus.cc:493] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:16.183044 1878 raft_consensus.cc:515] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:16.183735 1878 leader_election.cc:290] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:16.184756 991 leader_election.cc:343] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 003cd4472b424aacbba8c5f89900e228
I20260505 07:54:16.184924 2268 ts_tablet_manager.cc:1916] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.187209 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:16.187139 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f840d5f58161468385d5c4f116b78c3d" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.187487 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:16.187799 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:16.188149 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:16.188315 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:16.188592 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:16.188891 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:16.189785 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 132) took 8406 ms. Trace:
I20260505 07:54:16.189877 2276 rpcz_store.cc:276] 0505 07:54:07.783784 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.802183 (+ 18399us) service_pool.cc:224] Handling call
0505 07:54:16.157142 (+8354959us) ts_tablet_manager.cc:1151] Deleting tablet abe5f6f0774545b296bf1b03a08e145d
0505 07:54:16.157150 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.160832 (+ 3682us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.181234 (+ 20402us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.189768 (+ 8534us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8354864}
I20260505 07:54:16.190022 1181 tablet_bootstrap.cc:654] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.190200 2276 tablet_replica.cc:333] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.190469 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet abe5f6f0774545b296bf1b03a08e145d (table default.Table_c22dca26148c48f882eeea850f50bea4 [id=6c12928213864a1195f14e35d819f339]) successfully deleted
I20260505 07:54:16.190394 2276 raft_consensus.cc:2243] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:16.190971 854 leader_election.cc:343] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.191110 854 leader_election.cc:304] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.191620 2505 raft_consensus.cc:2749] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:16.190994 2276 raft_consensus.cc:2272] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.195191 2276 ts_tablet_manager.cc:1916] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.196200 1878 raft_consensus.cc:493] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.196359 1878 raft_consensus.cc:515] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.197059 1878 leader_election.cc:290] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:16.197686 991 leader_election.cc:343] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 27bacc97d7534284970f218cf68ac75b
I20260505 07:54:16.189230 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:16.200569 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:16.200822 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:16.201304 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:16.201638 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:16.201876 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:16.202489 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:16.203338 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:16.203464 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:16.204026 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bf0ce3e6c00541569d15c1b6ef347579 (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:16.204377 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3a73dbb979344805876efbafbf1b2985 (table default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f]). Expected version 1 got 0
I20260505 07:54:16.204593 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:16.204816 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:16.205072 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:16.205463 1354 catalog_manager.cc:5671] T ed0d43d85fcd4d05abf10d415e8dafbe P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:16.205919 2505 consensus_queue.cc:1048] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:16.209215 2268 ts_tablet_manager.cc:1929] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.209352 2268 log.cc:1199] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/f1058fcb9fa6457a85489049ca1d9ada
I20260505 07:54:16.209692 2268 ts_tablet_manager.cc:1950] T f1058fcb9fa6457a85489049ca1d9ada P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.211985 2276 ts_tablet_manager.cc:1929] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260505 07:54:16.212073 2276 log.cc:1199] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/4ba2cc218300472cb5fd6dc8cfb7811c
I20260505 07:54:16.212352 2276 ts_tablet_manager.cc:1950] T 4ba2cc218300472cb5fd6dc8cfb7811c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.213147 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 168) took 6335 ms. Trace:
I20260505 07:54:16.213231 2268 rpcz_store.cc:276] 0505 07:54:09.877625 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.877947 (+ 322us) service_pool.cc:224] Handling call
0505 07:54:16.181949 (+6304002us) ts_tablet_manager.cc:1151] Deleting tablet f1058fcb9fa6457a85489049ca1d9ada
0505 07:54:16.181960 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.191824 (+ 9864us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.209140 (+ 17316us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.213130 (+ 3990us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6303833}
I20260505 07:54:16.213609 2268 tablet_replica.cc:333] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.213781 2268 raft_consensus.cc:2243] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.213740 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet f1058fcb9fa6457a85489049ca1d9ada (table default.Table_2872bda04bf547648372a8f4bf5dc3ce [id=42ae40c0861e4489b724c9b8e3b590d7]) successfully deleted
I20260505 07:54:16.213896 2268 raft_consensus.cc:2272] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:16.215250 991 consensus_peers.cc:597] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4ba2cc218300472cb5fd6dc8cfb7811c. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:16.214574 2268 ts_tablet_manager.cc:1916] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.217325 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 88) took 8449 ms. Trace:
I20260505 07:54:16.217463 1181 tablet_bootstrap.cc:492] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.217408 2276 rpcz_store.cc:276] 0505 07:54:07.768221 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.768262 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:16.190153 (+8421891us) ts_tablet_manager.cc:1151] Deleting tablet 4ba2cc218300472cb5fd6dc8cfb7811c
0505 07:54:16.190162 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.207380 (+ 17218us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.211949 (+ 4569us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.217307 (+ 5358us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8387826}
I20260505 07:54:16.217581 1181 ts_tablet_manager.cc:1403] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.040s user 0.002s sys 0.003s
I20260505 07:54:16.217703 2276 tablet_replica.cc:333] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.217897 2276 raft_consensus.cc:2243] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.218009 2276 raft_consensus.cc:2272] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.217993 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 4ba2cc218300472cb5fd6dc8cfb7811c (table default.Table_27c52b9b060d4dd5a80799a987d5b7c9 [id=c41c8ca8836b4b4d96bc2f1129279f06]) successfully deleted
I20260505 07:54:16.218338 1181 raft_consensus.cc:359] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.218466 2276 ts_tablet_manager.cc:1916] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.218483 1181 raft_consensus.cc:385] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.218529 1181 raft_consensus.cc:740] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.218976 1181 consensus_queue.cc:260] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.219620 1181 ts_tablet_manager.cc:1434] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:16.219617 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:16.219950 1181 tablet_bootstrap.cc:492] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.220182 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:16.220368 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:16.220623 1354 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:16.221660 1878 raft_consensus.cc:493] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.221799 1181 tablet_bootstrap.cc:654] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.221812 1878 raft_consensus.cc:515] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.222652 1878 leader_election.cc:290] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.223006 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5083f98e8e14ccf99a19cdf2bc59018" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:16.223150 921 raft_consensus.cc:2468] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.223466 991 leader_election.cc:304] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:16.223742 1878 raft_consensus.cc:2804] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:16.223865 1878 raft_consensus.cc:493] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:16.223945 1878 raft_consensus.cc:3060] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.224540 1351 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e] alter complete (version 2)
I20260505 07:54:16.227396 1878 raft_consensus.cc:515] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.229466 1358 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet ba2494780bdd4b6dae6fbdf71125da88
I20260505 07:54:16.229966 1358 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 182) took 3959 ms. Trace:
I20260505 07:54:16.230058 1358 rpcz_store.cc:276] 0505 07:54:12.270955 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.597987 (+2327032us) service_pool.cc:224] Handling call
0505 07:54:14.598381 (+ 394us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:14.598527 (+ 146us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:14.598583 (+ 56us) catalog_manager.cc:2824] Locking tablets
0505 07:54:14.643032 (+ 44449us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:14.759816 (+116784us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:14.879723 (+119907us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:16.229425 (+1349702us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:16.229918 (+ 493us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:14.728752 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:14.728830 (+ 78us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:14.728835 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:14.728838 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:14.728980 (+ 142us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:14.728990 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:14.728994 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:14.729091 (+ 97us) tablet.cc:701] Row locks acquired
0505 07:54:14.729094 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:14.729128 (+ 34us) write_op.cc:270] Start()
0505 07:54:14.729154 (+ 26us) write_op.cc:276] Timestamp: P: 1777967654729064 usec, L: 0
0505 07:54:14.729158 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:14.729299 (+ 141us) log.cc:844] Serialized 759 byte log entry
0505 07:54:14.758477 (+ 29178us) op_driver.cc:464] REPLICATION: finished
0505 07:54:14.759450 (+ 973us) write_op.cc:301] APPLY: starting
0505 07:54:14.759460 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:14.759490 (+ 30us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:14.759512 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:14.759569 (+ 57us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:14.759611 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:14.759619 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:14.759686 (+ 67us) log.cc:844] Serialized 29 byte log entry
0505 07:54:14.759728 (+ 42us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.759740 (+ 12us) write_op.cc:454] Released schema lock
0505 07:54:14.759746 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:14.770772 (+ 11026us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x6170000ba5f0. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65b9eceb8 00007fd65b9f6bd1 00007fd65b9f68bd 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac 00007fd64fbf5528
Metrics: {"child_traces":[["op",{"apply.queue_time_us":945,"apply.run_cpu_time_us":352,"apply.run_wall_time_us":352,"num_ops":2,"prepare.queue_time_us":84829,"prepare.run_cpu_time_us":631,"prepare.run_wall_time_us":630,"raft.queue_time_us":7706,"raft.run_cpu_time_us":703,"raft.run_wall_time_us":33721,"replication_time_us":29306,"spinlock_wait_cycles":77053696,"thread_start_us":13531,"threads_started":1}]]}
I20260505 07:54:16.231967 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5083f98e8e14ccf99a19cdf2bc59018" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:16.232146 907 raft_consensus.cc:3060] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.234668 907 raft_consensus.cc:2468] T d5083f98e8e14ccf99a19cdf2bc59018 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:16.235267 760 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2494780bdd4b6dae6fbdf71125da88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:14 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.239728 899 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2494780bdd4b6dae6fbdf71125da88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:14 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.229338 1878 leader_election.cc:290] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.232935 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet ba2494780bdd4b6dae6fbdf71125da88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:14 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.241813 3100 tablet.cc:1724] T 423c22891bed462d89e9ec51a079f0f5 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260505 07:54:16.242501 3101 tablet.cc:1724] T 423c22891bed462d89e9ec51a079f0f5 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260505 07:54:16.240638 991 leader_election.cc:304] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:16.243845 1878 raft_consensus.cc:2804] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:16.244076 1878 raft_consensus.cc:697] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:16.244328 1878 consensus_queue.cc:237] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.248922 1352 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_e22a94851ca24c8997ab83e9509506d2 [id=50162ab861e0468d8622e6d2f33d9e79] alter complete (version 2)
I20260505 07:54:16.255635 2505 raft_consensus.cc:493] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.255805 2505 raft_consensus.cc:515] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.256531 2505 leader_election.cc:290] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.258003 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e243711164f04faabbde53b0a09d90b1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.259809 774 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e243711164f04faabbde53b0a09d90b1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:16.260324 854 leader_election.cc:343] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:16.265254 856 leader_election.cc:343] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.265357 856 leader_election.cc:304] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.265769 2505 raft_consensus.cc:2749] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:16.268889 1350 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 490483562d8f4c2daa7eaa333a9f3a0c consensus state has no leader
I20260505 07:54:16.281615 2276 ts_tablet_manager.cc:1929] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:16.281746 2276 log.cc:1199] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e8bf49b787ef4de095448751464708a4
I20260505 07:54:16.282049 2276 ts_tablet_manager.cc:1950] T e8bf49b787ef4de095448751464708a4 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.282872 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 89) took 8510 ms. Trace:
I20260505 07:54:16.282976 2276 rpcz_store.cc:276] 0505 07:54:07.772143 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.772197 (+ 54us) service_pool.cc:224] Handling call
0505 07:54:16.217659 (+8445462us) ts_tablet_manager.cc:1151] Deleting tablet e8bf49b787ef4de095448751464708a4
0505 07:54:16.217668 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.277928 (+ 60260us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.281555 (+ 3627us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.282848 (+ 1293us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8415194}
I20260505 07:54:16.283345 2276 tablet_replica.cc:333] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.283571 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e8bf49b787ef4de095448751464708a4 (table default.Table_b00ce8f351ce43edb2431c6e621d329e [id=39b432d60dc64a5aa01808a557238a4a]) successfully deleted
I20260505 07:54:16.287760 1048 raft_consensus.cc:3060] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.289369 2268 ts_tablet_manager.cc:1929] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.289474 2268 log.cc:1199] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/02e06d8815034d91986cef12964860ea
I20260505 07:54:16.289812 2268 ts_tablet_manager.cc:1950] T 02e06d8815034d91986cef12964860ea P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.290611 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 169) took 6412 ms. Trace:
I20260505 07:54:16.290683 2268 rpcz_store.cc:276] 0505 07:54:09.878298 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.888706 (+ 10408us) service_pool.cc:224] Handling call
0505 07:54:16.213570 (+6324864us) ts_tablet_manager.cc:1151] Deleting tablet 02e06d8815034d91986cef12964860ea
0505 07:54:16.213578 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.222687 (+ 9109us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.289315 (+ 66628us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.290593 (+ 1278us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6324651}
I20260505 07:54:16.291116 2268 tablet_replica.cc:333] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.291105 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 02e06d8815034d91986cef12964860ea (table default.Table_ca13a11f3fd249eabadc7213e7318acd [id=76602762055f42c0ad9805377eeb5d69]) successfully deleted
I20260505 07:54:16.291293 2268 raft_consensus.cc:2243] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:16.291538 2268 raft_consensus.cc:2272] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.292155 2268 ts_tablet_manager.cc:1916] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.283564 2276 raft_consensus.cc:2243] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.295209 2505 consensus_queue.cc:1048] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260505 07:54:16.296674 2276 raft_consensus.cc:2272] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.297333 2276 ts_tablet_manager.cc:1916] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.298730 3101 tablet.cc:1724] T f106a6d4855148fe83b1e90ea9f27d39 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:16.277879 1352 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3] alter complete (version 1)
I20260505 07:54:16.306906 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:16.318259 2856 raft_consensus.cc:493] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.318318 1878 raft_consensus.cc:493] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.318419 2856 raft_consensus.cc:515] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.318485 1878 raft_consensus.cc:515] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.319183 2856 leader_election.cc:290] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.320143 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d966bcd7f1419cbd090afc185ae78a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:16.321147 856 leader_election.cc:343] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.321626 1878 leader_election.cc:290] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.321952 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2706bc9b9e02414693062deff5cfe81b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:16.322106 912 raft_consensus.cc:2468] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.322494 991 leader_election.cc:304] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:16.322855 2650 raft_consensus.cc:2804] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:16.322999 2650 raft_consensus.cc:493] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:16.323132 2650 raft_consensus.cc:3060] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.325275 2872 raft_consensus.cc:493] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.325428 2872 raft_consensus.cc:515] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.326083 2872 leader_election.cc:290] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.327276 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2706bc9b9e02414693062deff5cfe81b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.328568 1357 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_925028e721e24ba68f31fe1040d4c2fd [id=0faa6676bab04919984b580299143271] alter complete (version 1)
W20260505 07:54:16.334903 717 consensus_peers.cc:597] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f3e076892234c1494b1a5c13cbfd2e2. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:16.335436 2650 raft_consensus.cc:515] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.335901 1050 raft_consensus.cc:2393] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:16.335988 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d966bcd7f1419cbd090afc185ae78a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.337643 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2706bc9b9e02414693062deff5cfe81b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:16.337858 912 raft_consensus.cc:3060] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.341436 912 raft_consensus.cc:2468] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:16.341940 991 leader_election.cc:304] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:16.344110 1878 raft_consensus.cc:2804] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:16.344154 899 tablet_service.cc:1558] Processing DeleteTablet for tablet abe5f6f0774545b296bf1b03a08e145d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.344380 1878 raft_consensus.cc:697] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:16.344801 1878 consensus_queue.cc:237] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:16.345124 854 leader_election.cc:343] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.345233 854 leader_election.cc:304] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.345449 2505 raft_consensus.cc:2749] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:16.345561 2268 ts_tablet_manager.cc:1929] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.350762 2505 raft_consensus.cc:493] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.350958 2505 raft_consensus.cc:515] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.351688 2505 leader_election.cc:290] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.353025 2268 log.cc:1199] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/f121fdd001ec4ea38ccd24a6afd9e51c
I20260505 07:54:16.352664 3100 tablet.cc:1724] T f106a6d4855148fe83b1e90ea9f27d39 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:16.352964 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5eaa91ecc1f43d18b313b6be4cc211e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:16.353917 856 leader_election.cc:343] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.347860 2650 leader_election.cc:290] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.354413 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5eaa91ecc1f43d18b313b6be4cc211e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:16.355626 854 leader_election.cc:343] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.355758 854 leader_election.cc:304] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.356338 2505 raft_consensus.cc:2749] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:16.360477 2268 ts_tablet_manager.cc:1950] T f121fdd001ec4ea38ccd24a6afd9e51c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.360909 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 170) took 6471 ms. Trace:
I20260505 07:54:16.361088 2268 rpcz_store.cc:276] 0505 07:54:09.889148 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.889528 (+ 380us) service_pool.cc:224] Handling call
0505 07:54:16.291072 (+6401544us) ts_tablet_manager.cc:1151] Deleting tablet f121fdd001ec4ea38ccd24a6afd9e51c
0505 07:54:16.291082 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.298100 (+ 7018us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.345509 (+ 47409us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.360894 (+ 15385us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6398492}
I20260505 07:54:16.361728 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 171) took 6471 ms. Trace:
I20260505 07:54:16.362056 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet f121fdd001ec4ea38ccd24a6afd9e51c (table default.Table_3a76cf09eaed4c9a8c267714a164dd66 [id=6be00fd9040d4febaef6b16dec369a3a]) successfully deleted
I20260505 07:54:16.362802 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c61afc74ec47fbac88d7237ce7d688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.362970 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a2e22cb61c340e289afd6cffbfc7661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.363427 2276 ts_tablet_manager.cc:1929] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.363538 2276 log.cc:1199] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/323269e708964848a94c64e6a781a860
I20260505 07:54:16.364004 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1456eecef5794452b8446596be57156e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.364392 2268 rpcz_store.cc:276] 0505 07:54:09.889835 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.892040 (+ 2205us) service_pool.cc:224] Handling call
0505 07:54:16.361605 (+6469565us) ts_tablet_manager.cc:1151] Deleting tablet 60772a9697ba417b879dc6025eeb8eec
0505 07:54:16.361616 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.361710 (+ 94us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6468856}
W20260505 07:54:16.371946 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 60772a9697ba417b879dc6025eeb8eec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60772a9697ba417b879dc6025eeb8eec already in progress: creating tablet
I20260505 07:54:16.376792 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2045202714c14aa388d1e8bc18e2063d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:16.377283 778 raft_consensus.cc:3060] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.377501 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bacc97d7534284970f218cf68ac75b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.377646 781 raft_consensus.cc:2468] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.378592 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5083f98e8e14ccf99a19cdf2bc59018" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.378681 2268 tablet_replica.cc:333] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.378772 781 raft_consensus.cc:2468] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.378859 2268 raft_consensus.cc:2243] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:16.379135 2268 raft_consensus.cc:2272] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.379288 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5083f98e8e14ccf99a19cdf2bc59018" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:16.379448 781 raft_consensus.cc:3060] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.379925 778 raft_consensus.cc:2468] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:16.380259 3004 raft_consensus.cc:493] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:16.380365 3004 raft_consensus.cc:515] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:16.380941 3004 leader_election.cc:290] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.382573 781 raft_consensus.cc:2468] T d5083f98e8e14ccf99a19cdf2bc59018 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:16.384235 2268 ts_tablet_manager.cc:1916] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.386853 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.387050 885 tablet_service.cc:1558] Processing DeleteTablet for tablet aab3850799254dda9716fd5bf9404159 with delete_type TABLET_DATA_DELETED (replaced by tablet 5006965919824adbbb23d744fb9ff335) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.387259 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 472441bd29704be59aabb0f7a4964bb0 with delete_type TABLET_DATA_DELETED (replaced by tablet e7887c3462f74740b314e3bec36e280d) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.387388 896 tablet_service.cc:1558] Processing DeleteTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.387475 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 080b2984fcf2478a8def6540da63554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.387856 885 tablet_service.cc:1558] Processing DeleteTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.387959 896 tablet_service.cc:1558] Processing DeleteTablet for tablet df1b62a30b2842b897a04c793bb5e7ce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.388334 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 297537a92684425d8508f7b3b4c9d379 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.388981 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ca2852cee84476dbe5e5072a43087a1 with delete_type TABLET_DATA_DELETED (replaced by tablet 01431ad112e8445487823efe8feb0dbe) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.389183 2276 ts_tablet_manager.cc:1950] T 323269e708964848a94c64e6a781a860 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.389416 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 727cacd242844455915bd9963d2db19f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.389518 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 90) took 8617 ms. Trace:
I20260505 07:54:16.389586 2276 rpcz_store.cc:276] 0505 07:54:07.772251 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.772324 (+ 73us) service_pool.cc:224] Handling call
0505 07:54:16.283288 (+8510964us) ts_tablet_manager.cc:1151] Deleting tablet 323269e708964848a94c64e6a781a860
0505 07:54:16.283301 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.337701 (+ 54400us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.363369 (+ 25668us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.389498 (+ 26129us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8480775}
I20260505 07:54:16.389846 2276 tablet_replica.cc:333] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
W20260505 07:54:16.398830 1357 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.2:57579 (request call id 159) took 2287 ms (client timeout 1000 ms). Trace:
I20260505 07:54:16.399319 896 tablet_service.cc:1558] Processing DeleteTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.399924 896 tablet_service.cc:1558] Processing DeleteTablet for tablet d622612bdc404727a27b83769bf39738 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:16.399016 1357 rpcz_store.cc:269] 0505 07:54:14.111369 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.533416 (+422047us) service_pool.cc:224] Handling call
0505 07:54:15.163681 (+630265us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.163689 (+ 8us) write_op.cc:454] Released schema lock
0505 07:54:16.398776 (+1235087us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.133448 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.133625 (+ 177us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.133634 (+ 9us) write_op.cc:435] Acquired schema lock
0505 07:54:15.133639 (+ 5us) tablet.cc:662] Decoding operations
0505 07:54:15.133892 (+ 253us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.133911 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.133918 (+ 7us) tablet.cc:685] Acquiring locks for 5 operations
0505 07:54:15.134169 (+ 251us) tablet.cc:701] Row locks acquired
0505 07:54:15.134173 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:15.134215 (+ 42us) write_op.cc:270] Start()
0505 07:54:15.134258 (+ 43us) write_op.cc:276] Timestamp: P: 1777967655134104 usec, L: 0
0505 07:54:15.134263 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.134501 (+ 238us) log.cc:844] Serialized 2024 byte log entry
0505 07:54:15.143855 (+ 9354us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.143965 (+ 110us) write_op.cc:301] APPLY: starting
0505 07:54:15.143980 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.144068 (+ 88us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.144101 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.144231 (+ 130us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.144265 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0505 07:54:15.144273 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:15.144356 (+ 83us) log.cc:844] Serialized 47 byte log entry
0505 07:54:15.144415 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.144438 (+ 23us) write_op.cc:454] Released schema lock
0505 07:54:15.144445 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:16.319542 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.319708 (+ 166us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.319716 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:16.319718 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:16.319833 (+ 115us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.319848 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.319852 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:16.319919 (+ 67us) tablet.cc:701] Row locks acquired
0505 07:54:16.319923 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:16.319961 (+ 38us) write_op.cc:270] Start()
0505 07:54:16.320009 (+ 48us) write_op.cc:276] Timestamp: P: 1777967656319866 usec, L: 0
0505 07:54:16.320012 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.320191 (+ 179us) log.cc:844] Serialized 356 byte log entry
0505 07:54:16.324947 (+ 4756us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.325045 (+ 98us) write_op.cc:301] APPLY: starting
0505 07:54:16.325059 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.325098 (+ 39us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.325123 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.325175 (+ 52us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.325205 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:16.325212 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:16.325277 (+ 65us) log.cc:844] Serialized 23 byte log entry
0505 07:54:16.325332 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.325349 (+ 17us) write_op.cc:454] Released schema lock
0505 07:54:16.325355 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.325416 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.325419 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":31592,"reported_tablets":252,"child_traces":[["op",{"apply.queue_time_us":64,"apply.run_cpu_time_us":555,"apply.run_wall_time_us":554,"num_ops":5,"prepare.queue_time_us":5300,"prepare.run_cpu_time_us":1187,"prepare.run_wall_time_us":1187,"raft.queue_time_us":1296,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":392,"replication_time_us":9560}],["op",{"apply.queue_time_us":53,"apply.run_cpu_time_us":364,"apply.run_wall_time_us":364,"num_ops":1,"prepare.queue_time_us":7922,"prepare.run_cpu_time_us":776,"prepare.run_wall_time_us":775,"raft.queue_time_us":616,"raft.run_cpu_time_us":559,"raft.run_wall_time_us":557,"replication_time_us":4909,"spinlock_wait_cycles":42880}]]}
I20260505 07:54:16.402058 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ad8a79245ad47738d079752e51aed2b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.402608 896 tablet_service.cc:1558] Processing DeleteTablet for tablet f2f5b72f58a24db0b58c15b47227bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.402977 896 tablet_service.cc:1558] Processing DeleteTablet for tablet e010f23c4df647ab863e03c3cfed7898 with delete_type TABLET_DATA_DELETED (replaced by tablet 09019fdf60374b11bf3427f21a55b5e0) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.403343 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 1602ccbae24544c4a6cf14ea073880b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.403725 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 6546745e894545ff9c5f91bcc6e1a41e with delete_type TABLET_DATA_DELETED (replaced by tablet 9c4fc18f6faa488882d9bffb30793cdd) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.404314 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 27bacc97d7534284970f218cf68ac75b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.404750 2650 raft_consensus.cc:493] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.404892 2650 raft_consensus.cc:515] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.405548 2650 leader_election.cc:290] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.406198 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 3075e87e078f4d2aa1d329d18902354b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:16.406466 1353 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.2:57579 (request call id 158) took 3408 ms (client timeout 1000 ms). Trace:
I20260505 07:54:16.406553 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:16.406584 1353 rpcz_store.cc:269] 0505 07:54:12.998271 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.410603 (+412332us) service_pool.cc:224] Handling call
0505 07:54:16.403946 (+2993343us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x6130001b02e0. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd656b2eecd 00007fd65980aeb9 00007fd6638c8803 00007fd663816705 00007fd66382d90d 00007fd6639d34e7 00007fd65a467ca6 00007fd65a4679d5 00007fd656bc310e 00007fd656bc1fee
0505 07:54:16.406396 (+ 2450us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26711,"reported_tablets":257,"spinlock_wait_cycles":38831488}
I20260505 07:54:16.406939 1353 master_service.cc:954] ReplaceTablet: received request to replace tablet bf0ce3e6c00541569d15c1b6ef347579 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:16.408507 896 tablet_service.cc:1558] Processing DeleteTablet for tablet e8bf49b787ef4de095448751464708a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.409406 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 323269e708964848a94c64e6a781a860 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]) successfully deleted
I20260505 07:54:16.410527 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abaa596871bb4a82aefc317932e91d35" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:16.410514 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2045202714c14aa388d1e8bc18e2063d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.410703 918 raft_consensus.cc:2468] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.410917 772 raft_consensus.cc:2376] T 2045202714c14aa388d1e8bc18e2063d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1. Re-sending same reply.
I20260505 07:54:16.412182 991 leader_election.cc:304] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:16.413466 3004 raft_consensus.cc:493] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.413575 3004 raft_consensus.cc:515] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.414117 3004 leader_election.cc:290] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.416379 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4515a0ff8304c69b9aa030e9b14d43f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.416579 1050 raft_consensus.cc:2468] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:16.416863 2268 ts_tablet_manager.cc:1929] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.417104 2268 log.cc:1199] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/7001482d90364dbdb8b9c5bfb98e1cd4
I20260505 07:54:16.417090 717 leader_election.cc:304] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:16.417663 2268 ts_tablet_manager.cc:1950] T 7001482d90364dbdb8b9c5bfb98e1cd4 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.418100 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 136) took 6803 ms. Trace:
I20260505 07:54:16.418257 2268 rpcz_store.cc:276] 0505 07:54:09.614316 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.614362 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:16.378626 (+6764264us) ts_tablet_manager.cc:1151] Deleting tablet 7001482d90364dbdb8b9c5bfb98e1cd4
0505 07:54:16.378641 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.392068 (+ 13427us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.416810 (+ 24742us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.418084 (+ 1274us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6484638}
I20260505 07:54:16.418802 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 172) took 6526 ms. Trace:
I20260505 07:54:16.418973 2268 rpcz_store.cc:276] 0505 07:54:09.892347 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.895723 (+ 3376us) service_pool.cc:224] Handling call
0505 07:54:16.418674 (+6522951us) ts_tablet_manager.cc:1151] Deleting tablet e5eaa91ecc1f43d18b313b6be4cc211e
0505 07:54:16.418684 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.418785 (+ 101us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6522782}
I20260505 07:54:16.422580 2831 raft_consensus.cc:2804] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:16.422675 2831 raft_consensus.cc:493] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:16.422758 2831 raft_consensus.cc:3060] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.426440 2831 raft_consensus.cc:515] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.427119 2831 leader_election.cc:290] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.428086 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4515a0ff8304c69b9aa030e9b14d43f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:16.431275 1050 raft_consensus.cc:3060] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.434281 1050 raft_consensus.cc:2468] T b4515a0ff8304c69b9aa030e9b14d43f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:16.434665 717 leader_election.cc:304] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:16.434991 3004 raft_consensus.cc:2804] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:16.435148 3004 raft_consensus.cc:697] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:16.435443 3004 consensus_queue.cc:237] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.437021 2268 tablet_replica.cc:333] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.437750 2268 raft_consensus.cc:2243] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.438253 2268 raft_consensus.cc:2272] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:16.439302 717 leader_election.cc:343] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: a0f2f46d0e7840458cb992ce2a2dbea2
I20260505 07:54:16.439702 2268 ts_tablet_manager.cc:1916] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:16.443272 3096 log.cc:927] Time spent T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c: Append to log took a long time: real 0.148s user 0.000s sys 0.000s
I20260505 07:54:16.444831 2650 raft_consensus.cc:2804] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:16.444924 2650 raft_consensus.cc:493] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:16.443897 2856 raft_consensus.cc:493] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.444998 2650 raft_consensus.cc:3060] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.445106 2856 raft_consensus.cc:515] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.445904 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d896e290167b4d0081e8e8b831c8ae2a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:16.446311 854 leader_election.cc:343] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.446601 2856 leader_election.cc:290] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.448709 2650 raft_consensus.cc:515] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.449514 2650 leader_election.cc:290] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.453894 2276 raft_consensus.cc:2243] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.453878 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d896e290167b4d0081e8e8b831c8ae2a" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.454051 2276 raft_consensus.cc:2272] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.454555 2276 ts_tablet_manager.cc:1916] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.460848 2276 ts_tablet_manager.cc:1929] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:16.460975 2276 log.cc:1199] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/60772a9697ba417b879dc6025eeb8eec
I20260505 07:54:16.461284 2276 ts_tablet_manager.cc:1950] T 60772a9697ba417b879dc6025eeb8eec P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.461547 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 91) took 8689 ms. Trace:
I20260505 07:54:16.461627 2276 rpcz_store.cc:276] 0505 07:54:07.772339 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.772458 (+ 119us) service_pool.cc:224] Handling call
0505 07:54:16.389809 (+8617351us) ts_tablet_manager.cc:1151] Deleting tablet 60772a9697ba417b879dc6025eeb8eec
0505 07:54:16.389816 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.457559 (+ 67743us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.460786 (+ 3227us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.461532 (+ 746us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63837,"tablet-delete.queue_time_us":8587261}
I20260505 07:54:16.461942 2276 tablet_replica.cc:333] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.462141 2276 raft_consensus.cc:2243] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.462296 2276 raft_consensus.cc:2272] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.462806 2276 ts_tablet_manager.cc:1916] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.463111 1878 raft_consensus.cc:493] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.463543 1355 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_4bd0f10670d240ee8aa468cf48521647 [id=3f62e5dc85a04f1295d1d2b84fff8f09] alter complete (version 1)
I20260505 07:54:16.465243 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 7001482d90364dbdb8b9c5bfb98e1cd4 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]) successfully deleted
I20260505 07:54:16.463490 1878 raft_consensus.cc:515] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.466638 1878 leader_election.cc:290] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.470242 2268 ts_tablet_manager.cc:1929] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:16.470330 2268 log.cc:1199] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/09b46ff0101a4e529077b9e30492ace7
I20260505 07:54:16.470635 2268 ts_tablet_manager.cc:1950] T 09b46ff0101a4e529077b9e30492ace7 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.471309 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 173) took 6575 ms. Trace:
I20260505 07:54:16.471378 2268 rpcz_store.cc:276] 0505 07:54:09.896089 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.896428 (+ 339us) service_pool.cc:224] Handling call
0505 07:54:16.436978 (+6540550us) ts_tablet_manager.cc:1151] Deleting tablet 09b46ff0101a4e529077b9e30492ace7
0505 07:54:16.436988 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.467285 (+ 30297us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.470204 (+ 2919us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.471292 (+ 1088us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6540407}
I20260505 07:54:16.473783 1172 tablet_bootstrap.cc:492] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:16.474036 1172 ts_tablet_manager.cc:1403] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.366s user 0.002s sys 0.003s
I20260505 07:54:16.474931 1172 raft_consensus.cc:359] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.475198 1172 raft_consensus.cc:385] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.475306 1172 raft_consensus.cc:740] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.475677 1172 consensus_queue.cc:260] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.476595 1172 ts_tablet_manager.cc:1434] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:16.477008 1172 tablet_bootstrap.cc:492] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:16.477560 1350 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 181) took 4204 ms. Trace:
I20260505 07:54:16.477753 1350 rpcz_store.cc:276] 0505 07:54:12.262229 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.536763 (+2274534us) service_pool.cc:224] Handling call
0505 07:54:14.537174 (+ 411us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:14.537403 (+ 229us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:14.537659 (+ 256us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:14.537754 (+ 95us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:14.694053 (+156299us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:16.466659 (+1772606us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:14.624130 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:14.624294 (+ 164us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:14.624360 (+ 66us) write_op.cc:435] Acquired schema lock
0505 07:54:14.624364 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:14.624622 (+ 258us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:14.624636 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:14.624642 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:14.624727 (+ 85us) tablet.cc:701] Row locks acquired
0505 07:54:14.624729 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:14.624767 (+ 38us) write_op.cc:270] Start()
0505 07:54:14.624808 (+ 41us) write_op.cc:276] Timestamp: P: 1777967654624715 usec, L: 0
0505 07:54:14.624813 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:14.625010 (+ 197us) log.cc:844] Serialized 405 byte log entry
0505 07:54:14.631554 (+ 6544us) op_driver.cc:464] REPLICATION: finished
0505 07:54:14.690884 (+ 59330us) write_op.cc:301] APPLY: starting
0505 07:54:14.690906 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:14.690956 (+ 50us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:14.690995 (+ 39us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:14.691045 (+ 50us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:14.691082 (+ 37us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:14.691093 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:14.691222 (+ 129us) log.cc:844] Serialized 23 byte log entry
0505 07:54:14.691463 (+ 241us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.691484 (+ 21us) write_op.cc:454] Released schema lock
0505 07:54:14.691493 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:14.691880 (+ 387us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.691885 (+ 5us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":59214,"apply.run_cpu_time_us":541,"apply.run_wall_time_us":703,"num_ops":1,"prepare.queue_time_us":85793,"prepare.run_cpu_time_us":929,"prepare.run_wall_time_us":64267,"raft.queue_time_us":113,"raft.run_cpu_time_us":574,"raft.run_wall_time_us":573,"replication_time_us":6704,"thread_start_us":173411,"threads_started":2}]]}
W20260505 07:54:16.479187 856 leader_election.cc:343] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.479338 856 leader_election.cc:304] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.479578 2856 raft_consensus.cc:2749] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:16.479723 751 tablet_service.cc:1511] Processing CreateTablet for tablet 025c94c839484adba26945b62a217f5c (DEFAULT_TABLE table=default.Table_e13508677ec64468adda12ac9efaeb05 [id=327b63fb4f744980a47bdce7db242629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.480233 751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025c94c839484adba26945b62a217f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.482317 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abaa596871bb4a82aefc317932e91d35" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:16.482455 918 raft_consensus.cc:3060] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.482687 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbfb2345a1a4b218c2dddbae6667e47" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:16.482875 912 raft_consensus.cc:2468] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.485482 918 raft_consensus.cc:2468] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:16.488505 749 tablet_service.cc:1511] Processing CreateTablet for tablet dc5fe0b4e2204e6499fa41d83d8dacec (DEFAULT_TABLE table=default.Table_48ab1b77fb0544cf81d96c1b8bd9a997 [id=8fd4517d64e0476891d81cd57115d865]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.488962 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5fe0b4e2204e6499fa41d83d8dacec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.489568 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 60772a9697ba417b879dc6025eeb8eec (table default.Table_ee5371f299e44106859f3663fe6052b7 [id=a4d53a8c6539419daba67a153292bb0c]) successfully deleted
I20260505 07:54:16.490293 760 tablet_service.cc:1511] Processing CreateTablet for tablet 66f994bc4c864ad2a04e7c0ac829a444 (DEFAULT_TABLE table=default.Table_da1e2e5149fb4ffba207cdfa95973244 [id=b3289842466f46e1a92f8e0ff21c87ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.490559 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f994bc4c864ad2a04e7c0ac829a444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.493214 2831 raft_consensus.cc:493] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.493331 2831 raft_consensus.cc:515] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.493938 2831 leader_election.cc:290] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.496191 2276 ts_tablet_manager.cc:1929] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.496284 2276 log.cc:1199] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/0baf83b155494fddb3c25144608706f7
I20260505 07:54:16.496605 2276 ts_tablet_manager.cc:1950] T 0baf83b155494fddb3c25144608706f7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.496862 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 92) took 8724 ms. Trace:
I20260505 07:54:16.496946 2276 rpcz_store.cc:276] 0505 07:54:07.772461 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.772587 (+ 126us) service_pool.cc:224] Handling call
0505 07:54:16.461898 (+8689311us) ts_tablet_manager.cc:1151] Deleting tablet 0baf83b155494fddb3c25144608706f7
0505 07:54:16.461907 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.474102 (+ 12195us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.496149 (+ 22047us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.496850 (+ 701us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8659299}
I20260505 07:54:16.497234 2276 tablet_replica.cc:333] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.497393 2276 raft_consensus.cc:2243] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.497545 2276 raft_consensus.cc:2272] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.498014 2276 ts_tablet_manager.cc:1916] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.479200 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 174) took 6582 ms. Trace:
I20260505 07:54:16.502367 2268 rpcz_store.cc:276] 0505 07:54:09.896706 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.896978 (+ 272us) service_pool.cc:224] Handling call
0505 07:54:16.471587 (+6574609us) ts_tablet_manager.cc:1151] Deleting tablet d896e290167b4d0081e8e8b831c8ae2a
0505 07:54:16.471596 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.479181 (+ 7585us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6574485}
I20260505 07:54:16.503122 2268 tablet_replica.cc:333] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.503384 2268 raft_consensus.cc:2243] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.503558 2268 raft_consensus.cc:2272] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.504016 1353 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet bf0ce3e6c00541569d15c1b6ef347579
I20260505 07:54:16.504282 744 tablet_service.cc:1511] Processing CreateTablet for tablet a248d71645bc45beb873e459e7c6aafa (DEFAULT_TABLE table=default.Table_4c7d78e1cc634a52ba8de34b2f90b809 [id=47383f18e5814a548287ab1d94d83ce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.504779 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a248d71645bc45beb873e459e7c6aafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.505136 760 tablet_service.cc:1558] Processing DeleteTablet for tablet bf0ce3e6c00541569d15c1b6ef347579 with delete_type TABLET_DATA_DELETED (replaced by tablet 823cd847ae2b496ca7e99998297ba2fe) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:16.505985 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e5eaa91ecc1f43d18b313b6be4cc211e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e5eaa91ecc1f43d18b313b6be4cc211e already in progress: creating tablet
I20260505 07:54:16.506880 2831 raft_consensus.cc:493] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.506989 2831 raft_consensus.cc:515] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.507530 2831 leader_election.cc:290] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.508497 760 tablet_service.cc:1511] Processing CreateTablet for tablet 9c37e8de609141c2b871694b2873a746 (DEFAULT_TABLE table=default.Table_6b9d99bdf8624bba90ba61af22f36dc2 [id=6fa686898f454d2aa691814ad1284d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.508870 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c37e8de609141c2b871694b2873a746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.511511 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a82ed2b87c487aac84194f2dcb0d6b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.511713 1353 catalog_manager.cc:6741] ReplaceTablet: tablet bf0ce3e6c00541569d15c1b6ef347579 deleted and replaced by tablet 823cd847ae2b496ca7e99998297ba2fe
I20260505 07:54:16.511860 1353 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 195) took 3184 ms. Trace:
W20260505 07:54:16.512033 717 leader_election.cc:343] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.512462 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abaa596871bb4a82aefc317932e91d35" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:16.512847 717 leader_election.cc:343] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.513381 2268 ts_tablet_manager.cc:1916] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.516489 896 tablet_service.cc:1511] Processing CreateTablet for tablet 025c94c839484adba26945b62a217f5c (DEFAULT_TABLE table=default.Table_e13508677ec64468adda12ac9efaeb05 [id=327b63fb4f744980a47bdce7db242629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.516942 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025c94c839484adba26945b62a217f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.519680 1353 rpcz_store.cc:276] 0505 07:54:13.326974 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.406841 (+3079867us) service_pool.cc:224] Handling call
0505 07:54:16.511820 (+104979us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.478424 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.478523 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.478528 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:16.478530 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:16.478647 (+ 117us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.478661 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.478666 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:16.478739 (+ 73us) tablet.cc:701] Row locks acquired
0505 07:54:16.478741 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:16.478773 (+ 32us) write_op.cc:270] Start()
0505 07:54:16.478810 (+ 37us) write_op.cc:276] Timestamp: P: 1777967656478735 usec, L: 0
0505 07:54:16.478813 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.478996 (+ 183us) log.cc:844] Serialized 622 byte log entry
0505 07:54:16.486080 (+ 7084us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.486311 (+ 231us) write_op.cc:301] APPLY: starting
0505 07:54:16.486326 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.486368 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.486418 (+ 50us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.486518 (+ 100us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.486569 (+ 51us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:16.486575 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:16.486654 (+ 79us) log.cc:844] Serialized 29 byte log entry
0505 07:54:16.486718 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.486737 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:16.486743 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.486809 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.486812 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":177,"apply.run_cpu_time_us":493,"apply.run_wall_time_us":494,"num_ops":2,"prepare.queue_time_us":98,"prepare.run_cpu_time_us":678,"prepare.run_wall_time_us":677,"raft.queue_time_us":1255,"raft.run_cpu_time_us":543,"raft.run_wall_time_us":6620,"replication_time_us":7239}]]}
I20260505 07:54:16.520622 1353 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7d9b2722b99c42b39ed977c61d37623b" } new_table_name: "default.Table_5c2b133923064d5eb9eabc21fb62f098" modify_external_catalogs: true
I20260505 07:54:16.523800 760 tablet_service.cc:1511] Processing CreateTablet for tablet 2abbf277ca8a4659ac97256c1ed2dc1b (DEFAULT_TABLE table=default.Table_1d71a1e96bb3432b9441aa916af3a79c [id=0509b98f6d8649179914b81aabf29ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.524205 760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abbf277ca8a4659ac97256c1ed2dc1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.525640 749 tablet_service.cc:1511] Processing CreateTablet for tablet f1c93803be7a4b0b95f2d464daf291b6 (DEFAULT_TABLE table=default.Table_040e8b41aef74a32a3a48a5e12a29ba7 [id=244506fede2a4f6380e236eabb95f0cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.525969 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1c93803be7a4b0b95f2d464daf291b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.526727 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 0baf83b155494fddb3c25144608706f7 (table default.Table_5edc57ff557343629eb69ed0fe818ded [id=a92ad827e74542d98f9825d5aa198670]) successfully deleted
I20260505 07:54:16.527829 1350 master_service.cc:954] ReplaceTablet: received request to replace tablet 21b48208e68247e78c12f0fec7ba07dc from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:16.519544 2268 ts_tablet_manager.cc:1929] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:16.538266 2268 log.cc:1199] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/abaa596871bb4a82aefc317932e91d35
I20260505 07:54:16.539513 2856 raft_consensus.cc:493] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.539705 2856 raft_consensus.cc:515] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.540287 2856 leader_election.cc:290] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.540855 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01431ad112e8445487823efe8feb0dbe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:16.541378 854 leader_election.cc:343] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.541985 2650 raft_consensus.cc:493] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.542192 2650 raft_consensus.cc:515] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.542910 2650 leader_election.cc:290] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:16.552142 717 consensus_peers.cc:597] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d77ff90ec2ff4b6eb1795d149459c23a. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:16.552562 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 09b46ff0101a4e529077b9e30492ace7 (table default.Table_c12be1928760438fab7c135df431bc79 [id=d567651314004cf4b33a8dfc1ef30698]) successfully deleted
I20260505 07:54:16.553406 1026 tablet_service.cc:1511] Processing CreateTablet for tablet a248d71645bc45beb873e459e7c6aafa (DEFAULT_TABLE table=default.Table_4c7d78e1cc634a52ba8de34b2f90b809 [id=47383f18e5814a548287ab1d94d83ce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.553406 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 025c94c839484adba26945b62a217f5c (DEFAULT_TABLE table=default.Table_e13508677ec64468adda12ac9efaeb05 [id=327b63fb4f744980a47bdce7db242629]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.554047 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a248d71645bc45beb873e459e7c6aafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.554198 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 025c94c839484adba26945b62a217f5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.555778 1181 tablet_bootstrap.cc:492] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.556133 1181 ts_tablet_manager.cc:1403] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.336s user 0.009s sys 0.011s
I20260505 07:54:16.566203 2447 ts_tablet_manager.cc:1929] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.566390 2447 log.cc:1199] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/9a06f77255544d7dabb71a4f70ada809
I20260505 07:54:16.566879 2447 ts_tablet_manager.cc:1950] T 9a06f77255544d7dabb71a4f70ada809 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:16.577579 3100 tablet.cc:1724] T c837877946604ef29f95dac9dd449bf3 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:16.580907 1181 raft_consensus.cc:359] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.581220 1181 raft_consensus.cc:385] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.581489 1181 raft_consensus.cc:740] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.582780 1181 consensus_queue.cc:260] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.584952 895 tablet_service.cc:1558] Processing DeleteTablet for tablet bf0ce3e6c00541569d15c1b6ef347579 with delete_type TABLET_DATA_DELETED (replaced by tablet 823cd847ae2b496ca7e99998297ba2fe) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.584977 885 tablet_service.cc:1511] Processing CreateTablet for tablet dc5fe0b4e2204e6499fa41d83d8dacec (DEFAULT_TABLE table=default.Table_48ab1b77fb0544cf81d96c1b8bd9a997 [id=8fd4517d64e0476891d81cd57115d865]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.585335 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5fe0b4e2204e6499fa41d83d8dacec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.585453 895 tablet_service.cc:1511] Processing CreateTablet for tablet 2abbf277ca8a4659ac97256c1ed2dc1b (DEFAULT_TABLE table=default.Table_1d71a1e96bb3432b9441aa916af3a79c [id=0509b98f6d8649179914b81aabf29ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.585749 895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abbf277ca8a4659ac97256c1ed2dc1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.586951 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01431ad112e8445487823efe8feb0dbe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.585629 894 tablet_service.cc:1511] Processing CreateTablet for tablet f1c93803be7a4b0b95f2d464daf291b6 (DEFAULT_TABLE table=default.Table_040e8b41aef74a32a3a48a5e12a29ba7 [id=244506fede2a4f6380e236eabb95f0cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.587813 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1c93803be7a4b0b95f2d464daf291b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.587787 899 tablet_service.cc:1511] Processing CreateTablet for tablet 66f994bc4c864ad2a04e7c0ac829a444 (DEFAULT_TABLE table=default.Table_da1e2e5149fb4ffba207cdfa95973244 [id=b3289842466f46e1a92f8e0ff21c87ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.588060 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f994bc4c864ad2a04e7c0ac829a444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.591424 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet bf0ce3e6c00541569d15c1b6ef347579 with delete_type TABLET_DATA_DELETED (replaced by tablet 823cd847ae2b496ca7e99998297ba2fe) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.591650 1023 tablet_service.cc:1511] Processing CreateTablet for tablet dc5fe0b4e2204e6499fa41d83d8dacec (DEFAULT_TABLE table=default.Table_48ab1b77fb0544cf81d96c1b8bd9a997 [id=8fd4517d64e0476891d81cd57115d865]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.591650 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 9c37e8de609141c2b871694b2873a746 (DEFAULT_TABLE table=default.Table_6b9d99bdf8624bba90ba61af22f36dc2 [id=6fa686898f454d2aa691814ad1284d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.591802 892 tablet_service.cc:1511] Processing CreateTablet for tablet 9c37e8de609141c2b871694b2873a746 (DEFAULT_TABLE table=default.Table_6b9d99bdf8624bba90ba61af22f36dc2 [id=6fa686898f454d2aa691814ad1284d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.592075 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c37e8de609141c2b871694b2873a746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.592124 1019 tablet_service.cc:1511] Processing CreateTablet for tablet f6bfd55f39da4ba981ad2289c696865a (DEFAULT_TABLE table=default.Table_97a06a334aa7478996655bf7214eb4c1 [id=f9ea6f95b6764a3ba5925a51285580e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.592271 1034 tablet_service.cc:1511] Processing CreateTablet for tablet f1c93803be7a4b0b95f2d464daf291b6 (DEFAULT_TABLE table=default.Table_040e8b41aef74a32a3a48a5e12a29ba7 [id=244506fede2a4f6380e236eabb95f0cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.592470 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 2abbf277ca8a4659ac97256c1ed2dc1b (DEFAULT_TABLE table=default.Table_1d71a1e96bb3432b9441aa916af3a79c [id=0509b98f6d8649179914b81aabf29ff9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.595973 887 tablet_service.cc:1511] Processing CreateTablet for tablet f6bfd55f39da4ba981ad2289c696865a (DEFAULT_TABLE table=default.Table_97a06a334aa7478996655bf7214eb4c1 [id=f9ea6f95b6764a3ba5925a51285580e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.596310 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6bfd55f39da4ba981ad2289c696865a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.597797 2268 ts_tablet_manager.cc:1950] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.604858 896 tablet_service.cc:1511] Processing CreateTablet for tablet a248d71645bc45beb873e459e7c6aafa (DEFAULT_TABLE table=default.Table_4c7d78e1cc634a52ba8de34b2f90b809 [id=47383f18e5814a548287ab1d94d83ce8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.605300 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a248d71645bc45beb873e459e7c6aafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.613924 2856 raft_consensus.cc:493] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.614084 2856 raft_consensus.cc:515] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.614791 2856 leader_election.cc:290] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:16.622260 1352 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.1:36755 (request call id 146) took 3581 ms (client timeout 1000 ms). Trace:
W20260505 07:54:16.622450 1352 rpcz_store.cc:269] 0505 07:54:13.041139 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.426324 (+385185us) service_pool.cc:224] Handling call
0505 07:54:16.622213 (+3195889us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:14.695411 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:14.695503 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:14.695508 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:14.695510 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:14.695610 (+ 100us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:14.695619 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:14.695623 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:14.695681 (+ 58us) tablet.cc:701] Row locks acquired
0505 07:54:14.695684 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:14.695712 (+ 28us) write_op.cc:270] Start()
0505 07:54:14.727155 (+ 31443us) write_op.cc:276] Timestamp: P: 1777967654695655 usec, L: 0
0505 07:54:14.727160 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:14.728452 (+ 1292us) log.cc:844] Serialized 356 byte log entry
0505 07:54:14.728620 (+ 168us) spinlock_profiling.cc:243] Waited 1.14 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65bae8f22 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac
0505 07:54:14.758412 (+ 29792us) op_driver.cc:464] REPLICATION: finished
0505 07:54:14.758985 (+ 573us) write_op.cc:301] APPLY: starting
0505 07:54:14.759004 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:14.759037 (+ 33us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:14.759040 (+ 3us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:14.759084 (+ 44us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:14.759112 (+ 28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:14.759119 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:14.759188 (+ 69us) log.cc:844] Serialized 23 byte log entry
0505 07:54:14.759243 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.759257 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:14.759263 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:14.759388 (+ 125us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.759392 (+ 4us) write_op.cc:454] Released schema lock
Related trace 'op':
0505 07:54:16.266879 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.266980 (+ 101us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.266997 (+ 17us) write_op.cc:435] Acquired schema lock
0505 07:54:16.267000 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:16.267095 (+ 95us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.267108 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.267112 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:16.267172 (+ 60us) tablet.cc:701] Row locks acquired
0505 07:54:16.267174 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:16.267202 (+ 28us) write_op.cc:270] Start()
0505 07:54:16.267233 (+ 31us) write_op.cc:276] Timestamp: P: 1777967656267117 usec, L: 0
0505 07:54:16.267236 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.267387 (+ 151us) log.cc:844] Serialized 356 byte log entry
0505 07:54:16.273166 (+ 5779us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.273288 (+ 122us) write_op.cc:301] APPLY: starting
0505 07:54:16.273310 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.273363 (+ 53us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.273403 (+ 40us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.273467 (+ 64us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.273510 (+ 43us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:16.273521 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:16.273615 (+ 94us) log.cc:844] Serialized 23 byte log entry
0505 07:54:16.273681 (+ 66us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.273716 (+ 35us) write_op.cc:454] Released schema lock
0505 07:54:16.273724 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.273816 (+ 92us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.273822 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":103521,"reported_tablets":277,"child_traces":[["op",{"apply.queue_time_us":510,"apply.run_cpu_time_us":346,"apply.run_wall_time_us":400,"num_ops":1,"prepare.queue_time_us":132555,"prepare.run_cpu_time_us":851,"prepare.run_wall_time_us":33263,"raft.queue_time_us":2469,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":5909,"replication_time_us":31201,"spinlock_wait_cycles":2611584}],["op",{"apply.queue_time_us":64,"apply.run_cpu_time_us":515,"apply.run_wall_time_us":515,"num_ops":1,"prepare.queue_time_us":51,"prepare.run_cpu_time_us":616,"prepare.run_wall_time_us":616,"raft.queue_time_us":1070,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":339,"replication_time_us":5893}]]}
I20260505 07:54:16.625102 1352 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_5e1ab4e85d4b46d8a5d98434da77c9a8" } modify_external_catalogs: true
I20260505 07:54:16.625366 1352 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_5e1ab4e85d4b46d8a5d98434da77c9a8" } modify_external_catalogs: true
I20260505 07:54:16.625814 1181 ts_tablet_manager.cc:1434] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.069s user 0.002s sys 0.000s
W20260505 07:54:16.626389 989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260505 07:54:16.640446 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 47945.4us
I20260505 07:54:16.641274 991 leader_election.cc:304] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:16.641844 2778 raft_consensus.cc:2804] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:16.641932 2778 raft_consensus.cc:493] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:16.642000 2778 raft_consensus.cc:3060] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.645237 2778 raft_consensus.cc:515] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:16.645998 1355 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.1:36755 (request call id 147) took 1989 ms (client timeout 1000 ms). Trace:
I20260505 07:54:16.646512 2778 leader_election.cc:290] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:16.646618 1355 rpcz_store.cc:269] 0505 07:54:14.656387 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.704220 (+ 47833us) service_pool.cc:224] Handling call
0505 07:54:16.460548 (+1756328us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.460690 (+ 142us) write_op.cc:454] Released schema lock
0505 07:54:16.645760 (+185070us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.452024 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.452144 (+ 120us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.452150 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:16.452153 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:16.452258 (+ 105us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.452271 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.452276 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:16.452338 (+ 62us) tablet.cc:701] Row locks acquired
0505 07:54:16.452340 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:16.452373 (+ 33us) write_op.cc:270] Start()
0505 07:54:16.452407 (+ 34us) write_op.cc:276] Timestamp: P: 1777967656452338 usec, L: 0
0505 07:54:16.452410 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.452578 (+ 168us) log.cc:844] Serialized 356 byte log entry
0505 07:54:16.456754 (+ 4176us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.456847 (+ 93us) write_op.cc:301] APPLY: starting
0505 07:54:16.456862 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.456899 (+ 37us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.456922 (+ 23us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.456974 (+ 52us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.457005 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:16.457012 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:16.457076 (+ 64us) log.cc:844] Serialized 23 byte log entry
0505 07:54:16.457127 (+ 51us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.457143 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:16.457150 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":153117,"reported_tablets":270,"child_traces":[["op",{"apply.queue_time_us":53,"apply.run_cpu_time_us":345,"apply.run_wall_time_us":345,"num_ops":1,"prepare.queue_time_us":10899,"prepare.run_cpu_time_us":681,"prepare.run_wall_time_us":682,"raft.queue_time_us":1997,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":453,"replication_time_us":4318}]]}
I20260505 07:54:16.659756 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 66f994bc4c864ad2a04e7c0ac829a444 (DEFAULT_TABLE table=default.Table_da1e2e5149fb4ffba207cdfa95973244 [id=b3289842466f46e1a92f8e0ff21c87ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.660264 1053 raft_consensus.cc:3060] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:16.664115 856 leader_election.cc:343] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.665002 856 leader_election.cc:304] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.665349 2505 raft_consensus.cc:2749] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:16.665666 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a3cd77808c4e858ea2ef9ed2a28c38" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.666318 991 leader_election.cc:304] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
W20260505 07:54:16.666572 991 raft_consensus.cc:2686] T abaa596871bb4a82aefc317932e91d35 P d63a3e535f8e4dd4a88e447253405b77: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260505 07:54:16.667367 3100 tablet.cc:1724] T 66082a7a75e345639e7a268727922fbb P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:16.670392 2276 ts_tablet_manager.cc:1929] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.670495 2276 log.cc:1199] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/feeafd614b3e4281a8e45b950e699b63
I20260505 07:54:16.670819 2276 ts_tablet_manager.cc:1950] T feeafd614b3e4281a8e45b950e699b63 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.671800 2505 raft_consensus.cc:493] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.671999 2505 raft_consensus.cc:515] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.673465 2505 leader_election.cc:290] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.674165 1355 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_94e6f1ee4ab947d8bc9864ab0c8bd524" } modify_external_catalogs: true
I20260505 07:54:16.674381 1355 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_94e6f1ee4ab947d8bc9864ab0c8bd524" } modify_external_catalogs: true
I20260505 07:54:16.678584 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9c37e8de609141c2b871694b2873a746. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:16.679503 1349 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.3:45737 (request call id 161) took 2969 ms (client timeout 1000 ms). Trace:
W20260505 07:54:16.680047 1349 rpcz_store.cc:269] 0505 07:54:13.709895 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.528264 (+818369us) service_pool.cc:224] Handling call
0505 07:54:15.714177 (+1185913us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.714183 (+ 6us) write_op.cc:454] Released schema lock
0505 07:54:16.679466 (+965283us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.699946 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.700122 (+ 176us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.700132 (+ 10us) write_op.cc:435] Acquired schema lock
0505 07:54:15.700136 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:15.700288 (+ 152us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.700307 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.700314 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:15.700461 (+ 147us) tablet.cc:701] Row locks acquired
0505 07:54:15.700465 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:15.700502 (+ 37us) write_op.cc:270] Start()
0505 07:54:15.700534 (+ 32us) write_op.cc:276] Timestamp: P: 1777967655700446 usec, L: 0
0505 07:54:15.700538 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.700761 (+ 223us) log.cc:844] Serialized 901 byte log entry
0505 07:54:15.706278 (+ 5517us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.706378 (+ 100us) write_op.cc:301] APPLY: starting
0505 07:54:15.706393 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.706439 (+ 46us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.706470 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.706539 (+ 69us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.706593 (+ 54us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:15.706600 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:15.706681 (+ 81us) log.cc:844] Serialized 29 byte log entry
0505 07:54:15.706737 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.706757 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:15.706764 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":114337,"reported_tablets":272,"child_traces":[["op",{"apply.queue_time_us":55,"apply.run_cpu_time_us":435,"apply.run_wall_time_us":435,"num_ops":2,"prepare.queue_time_us":53,"prepare.run_cpu_time_us":962,"prepare.run_wall_time_us":962,"raft.queue_time_us":1358,"raft.run_cpu_time_us":460,"raft.run_wall_time_us":460,"replication_time_us":5713}]]}
W20260505 07:54:16.681938 1348 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.3:45737 (request call id 162) took 1880 ms (client timeout 1000 ms). Trace:
W20260505 07:54:16.682019 1348 rpcz_store.cc:269] 0505 07:54:14.801367 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.872171 (+ 70804us) service_pool.cc:224] Handling call
0505 07:54:16.681874 (+1809703us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:15.878741 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:15.878925 (+ 184us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:15.878936 (+ 11us) write_op.cc:435] Acquired schema lock
0505 07:54:15.878940 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:15.879171 (+ 231us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:15.879191 (+ 20us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:15.879199 (+ 8us) tablet.cc:685] Acquiring locks for 3 operations
0505 07:54:15.879335 (+ 136us) tablet.cc:701] Row locks acquired
0505 07:54:15.879339 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:15.879382 (+ 43us) write_op.cc:270] Start()
0505 07:54:15.879427 (+ 45us) write_op.cc:276] Timestamp: P: 1777967655879329 usec, L: 0
0505 07:54:15.879433 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:15.879703 (+ 270us) log.cc:844] Serialized 1275 byte log entry
0505 07:54:15.884099 (+ 4396us) op_driver.cc:464] REPLICATION: finished
0505 07:54:15.884221 (+ 122us) write_op.cc:301] APPLY: starting
0505 07:54:15.884242 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:15.884304 (+ 62us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:15.884338 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:15.884441 (+ 103us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:15.884508 (+ 67us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0505 07:54:15.884520 (+ 12us) write_op.cc:312] APPLY: finished
0505 07:54:15.884636 (+ 116us) log.cc:844] Serialized 35 byte log entry
0505 07:54:15.884735 (+ 99us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.884763 (+ 28us) write_op.cc:454] Released schema lock
0505 07:54:15.884772 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:15.884852 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:15.884856 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":29910,"reported_tablets":267,"child_traces":[["op",{"apply.queue_time_us":65,"apply.run_cpu_time_us":626,"apply.run_wall_time_us":625,"num_ops":3,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":1105,"prepare.run_wall_time_us":1105,"raft.queue_time_us":1248,"raft.run_cpu_time_us":549,"raft.run_wall_time_us":548,"replication_time_us":4627}]]}
I20260505 07:54:16.682603 1348 master_service.cc:954] ReplaceTablet: received request to replace tablet ad5c570d4861460d8ab0c8ff8a7a3a03 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:16.680789 1349 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7001b868854949a5b9bee0ba45c2f522" } modify_external_catalogs: true
I20260505 07:54:16.683427 1349 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7001b868854949a5b9bee0ba45c2f522" } modify_external_catalogs: true
I20260505 07:54:16.686244 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc5fe0b4e2204e6499fa41d83d8dacec. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:16.689230 1351 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.3:45737 (request call id 160) took 4486 ms (client timeout 1000 ms). Trace:
I20260505 07:54:16.691779 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6bfd55f39da4ba981ad2289c696865a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:16.693653 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet d896e290167b4d0081e8e8b831c8ae2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d896e290167b4d0081e8e8b831c8ae2a already in progress: creating tablet
I20260505 07:54:16.694113 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef22101fe014e21adea6e10f4b550e7" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:16.694289 912 raft_consensus.cc:2468] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.695879 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1c93803be7a4b0b95f2d464daf291b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.699170 1348 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet ad5c570d4861460d8ab0c8ff8a7a3a03
I20260505 07:54:16.699779 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2abbf277ca8a4659ac97256c1ed2dc1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.699896 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 175) took 6802 ms. Trace:
I20260505 07:54:16.699995 1181 tablet_bootstrap.cc:492] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.699981 2268 rpcz_store.cc:276] 0505 07:54:09.897233 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.897502 (+ 269us) service_pool.cc:224] Handling call
0505 07:54:16.503056 (+6605554us) ts_tablet_manager.cc:1151] Deleting tablet abaa596871bb4a82aefc317932e91d35
0505 07:54:16.503072 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.516692 (+ 13620us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.519497 (+ 2805us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.699868 (+180371us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6605411}
I20260505 07:54:16.700354 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 176) took 6802 ms. Trace:
I20260505 07:54:16.700412 2268 rpcz_store.cc:276] 0505 07:54:09.897848 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.899552 (+ 1704us) service_pool.cc:224] Handling call
0505 07:54:16.700261 (+6800709us) ts_tablet_manager.cc:1151] Deleting tablet e8bf49b787ef4de095448751464708a4
0505 07:54:16.700269 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.700341 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6800534}
I20260505 07:54:16.700624 2268 tablet_replica.cc:333] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.700819 2268 raft_consensus.cc:2243] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:16.701058 2268 raft_consensus.cc:2272] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.701319 1181 tablet_bootstrap.cc:654] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.701530 2268 ts_tablet_manager.cc:1916] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.701704 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66f994bc4c864ad2a04e7c0ac829a444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.703842 1348 catalog_manager.cc:6741] ReplaceTablet: tablet ad5c570d4861460d8ab0c8ff8a7a3a03 deleted and replaced by tablet b53db8fdadad4ec09d05f65c84100ff0
I20260505 07:54:16.703945 1348 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 201) took 3011 ms. Trace:
I20260505 07:54:16.704025 1348 rpcz_store.cc:276] 0505 07:54:13.692364 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.682536 (+2990172us) service_pool.cc:224] Handling call
0505 07:54:16.703920 (+ 21384us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.689864 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.689962 (+ 98us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.689969 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:16.689971 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:16.690090 (+ 119us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.690103 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.690108 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:16.690217 (+ 109us) tablet.cc:701] Row locks acquired
0505 07:54:16.690231 (+ 14us) write_op.cc:260] PREPARE: finished
0505 07:54:16.690261 (+ 30us) write_op.cc:270] Start()
0505 07:54:16.690290 (+ 29us) write_op.cc:276] Timestamp: P: 1777967656690203 usec, L: 0
0505 07:54:16.690294 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.690444 (+ 150us) log.cc:844] Serialized 622 byte log entry
0505 07:54:16.694772 (+ 4328us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.697358 (+ 2586us) write_op.cc:301] APPLY: starting
0505 07:54:16.697378 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.697447 (+ 69us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.697476 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.697599 (+ 123us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.697678 (+ 79us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:16.697689 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:16.697776 (+ 87us) log.cc:844] Serialized 29 byte log entry
0505 07:54:16.697844 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.697869 (+ 25us) write_op.cc:454] Released schema lock
0505 07:54:16.697877 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.697951 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.697957 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":71,"apply.run_cpu_time_us":587,"apply.run_wall_time_us":586,"num_ops":2,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":697,"prepare.run_wall_time_us":698,"raft.queue_time_us":1007,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":396,"replication_time_us":4441}]]}
I20260505 07:54:16.705967 1348 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_c9015cf60d4c41b18743d55fbdcdf65e" } new_table_name: "default.Table_65d14038399640c284b8382aa80e344f" modify_external_catalogs: true
I20260505 07:54:16.708767 2268 ts_tablet_manager.cc:1929] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260505 07:54:16.708860 2268 log.cc:1199] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/4ba2cc218300472cb5fd6dc8cfb7811c
I20260505 07:54:16.709239 2268 ts_tablet_manager.cc:1950] T 4ba2cc218300472cb5fd6dc8cfb7811c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.709699 3100 tablet.cc:1724] T 76939b7978ee4f7ea0ad29312de2a31a P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:16.709967 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 177) took 6810 ms. Trace:
I20260505 07:54:16.710140 2268 rpcz_store.cc:276] 0505 07:54:09.899931 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.900283 (+ 352us) service_pool.cc:224] Handling call
0505 07:54:16.700586 (+6800303us) ts_tablet_manager.cc:1151] Deleting tablet 4ba2cc218300472cb5fd6dc8cfb7811c
0505 07:54:16.700594 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.704910 (+ 4316us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.708723 (+ 3813us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.709951 (+ 1228us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6800162}
I20260505 07:54:16.710402 2268 tablet_replica.cc:333] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.710589 2268 raft_consensus.cc:2243] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.710742 2268 raft_consensus.cc:2272] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:16.706508 1351 rpcz_store.cc:269] 0505 07:54:12.202293 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.383153 (+180860us) service_pool.cc:224] Handling call
0505 07:54:16.689199 (+4306046us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:13.507511 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:13.507701 (+ 190us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:13.507708 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:13.507710 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:13.507906 (+ 196us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:13.507915 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:13.507952 (+ 37us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:13.508173 (+ 221us) tablet.cc:701] Row locks acquired
0505 07:54:13.508176 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:13.508237 (+ 61us) write_op.cc:270] Start()
0505 07:54:13.508297 (+ 60us) write_op.cc:276] Timestamp: P: 1777967653508178 usec, L: 0
0505 07:54:13.508300 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:13.508687 (+ 387us) log.cc:844] Serialized 901 byte log entry
0505 07:54:13.716562 (+207875us) op_driver.cc:464] REPLICATION: finished
0505 07:54:13.725411 (+ 8849us) write_op.cc:301] APPLY: starting
0505 07:54:13.725436 (+ 25us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:13.725494 (+ 58us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:13.725535 (+ 41us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:13.725667 (+ 132us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:13.725713 (+ 46us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:13.725723 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:13.725828 (+ 105us) log.cc:844] Serialized 29 byte log entry
0505 07:54:13.726102 (+ 274us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.726125 (+ 23us) write_op.cc:454] Released schema lock
0505 07:54:13.726133 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:13.726212 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:13.726215 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
0505 07:54:14.688432 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:14.688531 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:14.688537 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:14.688539 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:14.688653 (+ 114us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:14.688665 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:14.688669 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:14.688754 (+ 85us) tablet.cc:701] Row locks acquired
0505 07:54:14.688757 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:14.688795 (+ 38us) write_op.cc:270] Start()
0505 07:54:14.688831 (+ 36us) write_op.cc:276] Timestamp: P: 1777967654688734 usec, L: 0
0505 07:54:14.688835 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:14.695124 (+ 6289us) log.cc:844] Serialized 356 byte log entry
0505 07:54:14.695327 (+ 203us) spinlock_profiling.cc:243] Waited 6.11 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65bae8f22 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac
0505 07:54:14.710854 (+ 15527us) op_driver.cc:464] REPLICATION: finished
0505 07:54:14.727544 (+ 16690us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x6190000a0258. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65d00c824 00007fd65ced3a91 00007fd65ced4bfa 00007fd65ced867c 00007fd65ced83a1 00007fd66393c9cd 00007fd65bb2127c 00007fd65baabe19 00007fd65baed427 00007fd65ba2e1ac
0505 07:54:14.727782 (+ 238us) write_op.cc:301] APPLY: starting
0505 07:54:14.727800 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:14.727839 (+ 39us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:14.727841 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:14.727890 (+ 49us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:14.727926 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:14.727933 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:14.728006 (+ 73us) log.cc:844] Serialized 23 byte log entry
0505 07:54:14.728075 (+ 69us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.728090 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:14.728095 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:14.728224 (+ 129us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:14.728230 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":140466,"reported_tablets":283,"child_traces":[["op",{"apply.queue_time_us":8727,"apply.run_cpu_time_us":614,"apply.run_wall_time_us":811,"num_ops":2,"prepare.queue_time_us":6401,"prepare.run_cpu_time_us":946,"prepare.run_wall_time_us":1716,"replication_time_us":208231}],["op",{"apply.queue_time_us":187,"apply.run_cpu_time_us":381,"apply.run_wall_time_us":436,"num_ops":1,"prepare.queue_time_us":135580,"prepare.run_cpu_time_us":953,"prepare.run_wall_time_us":6954,"raft.queue_time_us":447,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":14958,"replication_time_us":21993,"spinlock_wait_cycles":51768320}]]}
I20260505 07:54:16.715963 2268 ts_tablet_manager.cc:1916] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:16.721508 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 57402.8us
I20260505 07:54:16.724257 2247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260505 07:54:16.725039 1351 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_ad8e6e942c73439fb64536b3ab1edc9c" } new_table_name: "default.Table_eee3bd15017341fca226f671948e94e9" modify_external_catalogs: true
I20260505 07:54:16.725317 3101 tablet.cc:1724] T c837877946604ef29f95dac9dd449bf3 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:16.726745 892 tablet_service.cc:1558] Processing DeleteTablet for tablet ad5c570d4861460d8ab0c8ff8a7a3a03 with delete_type TABLET_DATA_DELETED (replaced by tablet b53db8fdadad4ec09d05f65c84100ff0) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.584017 3013 raft_consensus.cc:493] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.727100 3013 raft_consensus.cc:515] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.727864 3013 leader_election.cc:290] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.615450 771 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a3cd77808c4e858ea2ef9ed2a28c38" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.616784 760 tablet_service.cc:1558] Processing DeleteTablet for tablet 36b33d265ba04e93993c7e0a4efea1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.616936 751 tablet_service.cc:1558] Processing DeleteTablet for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.617488 757 tablet_service.cc:1558] Processing DeleteTablet for tablet 72da22d790624b07bfca6f6b6301c925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:07 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.618384 754 tablet_service.cc:1558] Processing DeleteTablet for tablet abaa596871bb4a82aefc317932e91d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.618868 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 323269e708964848a94c64e6a781a860 with delete_type TABLET_DATA_DELETED (replaced by tablet e010f23c4df647ab863e03c3cfed7898) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.619439 761 tablet_service.cc:1558] Processing DeleteTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.619912 743 tablet_service.cc:1558] Processing DeleteTablet for tablet 397d33e908af4a0bb8dde26d273be01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.620044 750 tablet_service.cc:1558] Processing DeleteTablet for tablet 78a6470720984761a8606abeafbef992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.620111 748 tablet_service.cc:1558] Processing DeleteTablet for tablet ccdde3e5a13e436d9f2328831e155adb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.620407 745 tablet_service.cc:1558] Processing DeleteTablet for tablet 202a237ca580482c98803683a77d96d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.620724 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 03f31efbc98c484c950c70d5e84d2136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.627890 3004 consensus_queue.cc:1048] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:16.634321 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2706bc9b9e02414693062deff5cfe81b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.729261 781 raft_consensus.cc:2468] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.635775 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abaa596871bb4a82aefc317932e91d35" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.729542 778 raft_consensus.cc:2468] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.635969 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abaa596871bb4a82aefc317932e91d35" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:16.729736 776 raft_consensus.cc:3060] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.729991 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efb07cb60f64dfead739a3ca1343df9" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.732368 776 raft_consensus.cc:2468] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:16.636142 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbfb2345a1a4b218c2dddbae6667e47" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.734076 2268 ts_tablet_manager.cc:1929] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:16.734163 2268 log.cc:1199] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/27bacc97d7534284970f218cf68ac75b
I20260505 07:54:16.734514 2268 ts_tablet_manager.cc:1950] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.636180 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2706bc9b9e02414693062deff5cfe81b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:16.637672 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef22101fe014e21adea6e10f4b550e7" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.738657 767 raft_consensus.cc:2468] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:16.738919 772 raft_consensus.cc:3060] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.739845 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 178) took 6839 ms. Trace:
I20260505 07:54:16.739966 2268 rpcz_store.cc:276] 0505 07:54:09.900560 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.900854 (+ 294us) service_pool.cc:224] Handling call
0505 07:54:16.710360 (+6809506us) ts_tablet_manager.cc:1151] Deleting tablet 27bacc97d7534284970f218cf68ac75b
0505 07:54:16.710370 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.718516 (+ 8146us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.734034 (+ 15518us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.739825 (+ 5791us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6809369}
I20260505 07:54:16.740332 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 179) took 6839 ms. Trace:
I20260505 07:54:16.740408 2268 rpcz_store.cc:276] 0505 07:54:09.901132 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.901436 (+ 304us) service_pool.cc:224] Handling call
0505 07:54:16.740244 (+6838808us) ts_tablet_manager.cc:1151] Deleting tablet 8eb7eaff8837485a8ae3ccb44c60471d
0505 07:54:16.740253 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.740318 (+ 65us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6838670}
I20260505 07:54:16.740538 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef22101fe014e21adea6e10f4b550e7" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.741097 1052 raft_consensus.cc:2468] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:16.741604 717 leader_election.cc:304] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:16.741832 3004 raft_consensus.cc:2804] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:16.741923 3004 raft_consensus.cc:493] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:16.741982 3004 raft_consensus.cc:3060] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.740664 2268 tablet_replica.cc:333] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.743134 2268 raft_consensus.cc:2243] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:16.743392 2268 pending_rounds.cc:70] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:54:16.743435 2268 pending_rounds.cc:77] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7282555515118624768 op_type: NO_OP noop_request { }
I20260505 07:54:16.743518 2268 raft_consensus.cc:2889] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:16.743628 2268 raft_consensus.cc:2272] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.744135 2268 ts_tablet_manager.cc:1916] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.744235 3004 raft_consensus.cc:515] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.744459 1181 tablet_bootstrap.cc:492] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.744602 1181 ts_tablet_manager.cc:1403] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.045s user 0.007s sys 0.002s
I20260505 07:54:16.744802 3004 leader_election.cc:290] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.745162 1181 raft_consensus.cc:359] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.745289 1181 raft_consensus.cc:385] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.745340 1181 raft_consensus.cc:740] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.745484 1181 consensus_queue.cc:260] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.745967 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef22101fe014e21adea6e10f4b550e7" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:16.746028 1181 ts_tablet_manager.cc:1434] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260505 07:54:16.746115 1048 raft_consensus.cc:3060] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.746258 1181 tablet_bootstrap.cc:492] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.747536 1181 tablet_bootstrap.cc:654] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.748663 1048 raft_consensus.cc:2468] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:16.749135 717 leader_election.cc:304] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:16.749456 3004 raft_consensus.cc:2804] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:16.749573 3004 raft_consensus.cc:697] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:16.749825 3004 consensus_queue.cc:237] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.754309 2268 ts_tablet_manager.cc:1929] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:16.754429 2268 log.cc:1199] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/78a6470720984761a8606abeafbef992
I20260505 07:54:16.754856 2268 ts_tablet_manager.cc:1950] T 78a6470720984761a8606abeafbef992 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.755460 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 180) took 6853 ms. Trace:
I20260505 07:54:16.755541 2268 rpcz_store.cc:276] 0505 07:54:09.901723 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.901973 (+ 250us) service_pool.cc:224] Handling call
0505 07:54:16.740622 (+6838649us) ts_tablet_manager.cc:1151] Deleting tablet 78a6470720984761a8606abeafbef992
0505 07:54:16.740631 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.746756 (+ 6125us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.754240 (+ 7484us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.755446 (+ 1206us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6838500}
I20260505 07:54:16.759953 2268 tablet_replica.cc:333] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.760195 2268 raft_consensus.cc:2243] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:16.760210 772 raft_consensus.cc:2468] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:16.760478 2268 raft_consensus.cc:2272] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.760741 1181 tablet_bootstrap.cc:492] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.760846 1181 ts_tablet_manager.cc:1403] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.000s
I20260505 07:54:16.760927 2268 ts_tablet_manager.cc:1916] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.761422 1181 raft_consensus.cc:359] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.761567 1181 raft_consensus.cc:385] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.761627 1181 raft_consensus.cc:740] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.761813 1181 consensus_queue.cc:260] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.762385 1181 ts_tablet_manager.cc:1434] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260505 07:54:16.762609 1181 tablet_bootstrap.cc:492] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.640215 3112 tablet.cc:1724] T f106a6d4855148fe83b1e90ea9f27d39 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:16.763866 1181 tablet_bootstrap.cc:654] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.768656 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:16.769061 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:16.668370 744 tablet_service.cc:1511] Processing CreateTablet for tablet f6bfd55f39da4ba981ad2289c696865a (DEFAULT_TABLE table=default.Table_97a06a334aa7478996655bf7214eb4c1 [id=f9ea6f95b6764a3ba5925a51285580e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.673187 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efb07cb60f64dfead739a3ca1343df9" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:16.683936 2831 consensus_queue.cc:1048] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:16.770010 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:16.699772 762 tablet_service.cc:1558] Processing DeleteTablet for tablet ad5c570d4861460d8ab0c8ff8a7a3a03 with delete_type TABLET_DATA_DELETED (replaced by tablet b53db8fdadad4ec09d05f65c84100ff0) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:16.717387 1176 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.569s user 0.020s sys 0.088s
W20260505 07:54:16.770138 1176 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.621s user 0.020s sys 0.088s
I20260505 07:54:16.770238 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:16.770416 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:16.770565 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:16.770701 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:16.770833 1176 tablet_bootstrap.cc:492] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:16.770952 1176 ts_tablet_manager.cc:1403] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.624s user 0.021s sys 0.088s
I20260505 07:54:16.771061 2268 ts_tablet_manager.cc:1929] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.771158 2268 log.cc:1199] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/202a237ca580482c98803683a77d96d2
I20260505 07:54:16.771471 2268 ts_tablet_manager.cc:1950] T 202a237ca580482c98803683a77d96d2 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.771703 1176 raft_consensus.cc:359] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.771862 1176 raft_consensus.cc:385] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.771907 1176 raft_consensus.cc:740] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.772055 1176 consensus_queue.cc:260] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.772486 1176 ts_tablet_manager.cc:1434] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260505 07:54:16.772763 1176 tablet_bootstrap.cc:492] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:16.773578 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6bfd55f39da4ba981ad2289c696865a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.774238 1176 tablet_bootstrap.cc:654] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.770879 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:16.774740 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:16.775403 1358 catalog_manager.cc:5671] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:16.775696 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 181) took 6873 ms. Trace:
I20260505 07:54:16.775789 2268 rpcz_store.cc:276] 0505 07:54:09.902530 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.903834 (+ 1304us) service_pool.cc:224] Handling call
0505 07:54:16.759902 (+6856068us) ts_tablet_manager.cc:1151] Deleting tablet 202a237ca580482c98803683a77d96d2
0505 07:54:16.759914 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.768180 (+ 8266us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.770999 (+ 2819us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.775678 (+ 4679us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6855895}
I20260505 07:54:16.776137 2268 tablet_replica.cc:333] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.776332 2268 raft_consensus.cc:2243] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.776453 2268 raft_consensus.cc:2272] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.776928 2268 ts_tablet_manager.cc:1916] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.777285 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:16.777814 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:16.783933 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:16.784394 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:16.784446 1176 tablet_bootstrap.cc:492] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:16.784566 1176 ts_tablet_manager.cc:1403] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.006s
I20260505 07:54:16.784883 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:16.785529 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:16.786317 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:16.787154 1358 catalog_manager.cc:5671] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:16.788250 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:16.789105 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:16.789839 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:16.790601 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:16.791757 2268 ts_tablet_manager.cc:1929] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.791862 2268 log.cc:1199] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/003cd4472b424aacbba8c5f89900e228
I20260505 07:54:16.792307 2268 ts_tablet_manager.cc:1950] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.785199 1176 raft_consensus.cc:359] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.792383 1176 raft_consensus.cc:385] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.792471 1176 raft_consensus.cc:740] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.791755 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:16.792743 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 183) took 6887 ms. Trace:
I20260505 07:54:16.792696 1176 consensus_queue.cc:260] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.792832 2268 rpcz_store.cc:276] 0505 07:54:09.905057 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.907731 (+ 2674us) service_pool.cc:224] Handling call
0505 07:54:16.776087 (+6868356us) ts_tablet_manager.cc:1151] Deleting tablet 003cd4472b424aacbba8c5f89900e228
0505 07:54:16.776099 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.782136 (+ 6037us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.791691 (+ 9555us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.792724 (+ 1033us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6868187}
I20260505 07:54:16.793135 2268 tablet_replica.cc:333] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.793346 2268 raft_consensus.cc:2243] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.793509 2268 raft_consensus.cc:2272] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.793758 1176 ts_tablet_manager.cc:1434] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260505 07:54:16.794067 2268 ts_tablet_manager.cc:1916] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.794080 1176 tablet_bootstrap.cc:492] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:16.794801 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:16.796121 1358 catalog_manager.cc:5671] T 66be7b6c33a342d6890e459d90d40583 P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:16.797653 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 175) took 7245 ms. Trace:
I20260505 07:54:16.798270 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:16.798839 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:16.799252 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:16.799782 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 490483562d8f4c2daa7eaa333a9f3a0c (table default.Table_3acf7c3eb13d473a9d952098c7524753 [id=fe7ee6e8118840e8a9f70a007d80c2b3]). Expected version 1 got 0
W20260505 07:54:16.803767 988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260505 07:54:16.803968 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:16.804898 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:16.805863 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:16.806214 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 93) took 9033 ms. Trace:
I20260505 07:54:16.806329 2276 rpcz_store.cc:276] 0505 07:54:07.772582 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.772728 (+ 146us) service_pool.cc:224] Handling call
0505 07:54:16.497185 (+8724457us) ts_tablet_manager.cc:1151] Deleting tablet feeafd614b3e4281a8e45b950e699b63
0505 07:54:16.497193 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.500777 (+ 3584us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.670343 (+169566us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.806189 (+135846us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8694559}
I20260505 07:54:16.806674 2276 tablet_replica.cc:333] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.807675 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:16.808521 1172 tablet_bootstrap.cc:700] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533: Time spent opening tablet: real 0.330s user 0.000s sys 0.001s
I20260505 07:54:16.808835 1172 tablet_bootstrap.cc:654] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.809190 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:16.809945 2276 raft_consensus.cc:2243] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:16.810274 2276 pending_rounds.cc:70] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:54:16.810328 2276 pending_rounds.cc:77] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7282555512635650048 op_type: NO_OP noop_request { }
I20260505 07:54:16.810431 2276 raft_consensus.cc:2889] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:16.810525 2276 raft_consensus.cc:2272] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.811045 2276 ts_tablet_manager.cc:1916] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.811461 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:16.812538 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:16.817339 2276 ts_tablet_manager.cc:1929] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:16.817436 2276 log.cc:1199] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/0459f2d45a2d44f1a73e0a0fff53866c
I20260505 07:54:16.817895 2276 ts_tablet_manager.cc:1950] T 0459f2d45a2d44f1a73e0a0fff53866c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.818181 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 94) took 9045 ms. Trace:
I20260505 07:54:16.818280 2276 rpcz_store.cc:276] 0505 07:54:07.772688 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.772837 (+ 149us) service_pool.cc:224] Handling call
0505 07:54:16.806633 (+9033796us) ts_tablet_manager.cc:1151] Deleting tablet 0459f2d45a2d44f1a73e0a0fff53866c
0505 07:54:16.806641 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.814657 (+ 8016us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.817299 (+ 2642us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.818162 (+ 863us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3035,"tablet-delete.queue_time_us":9003961}
I20260505 07:54:16.818598 2276 tablet_replica.cc:333] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.818776 2276 raft_consensus.cc:2243] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:16.819074 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:16.819124 2276 raft_consensus.cc:2272] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.819535 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:16.820688 2268 ts_tablet_manager.cc:1929] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.820796 2268 log.cc:1199] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/b0b9e8eb767a4eceb583ff9b47000d58
I20260505 07:54:16.821226 2268 ts_tablet_manager.cc:1950] T b0b9e8eb767a4eceb583ff9b47000d58 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.821753 1358 catalog_manager.cc:5671] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:16.822039 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 185) took 6912 ms. Trace:
I20260505 07:54:16.822134 2268 rpcz_store.cc:276] 0505 07:54:09.909119 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.910652 (+ 1533us) service_pool.cc:224] Handling call
0505 07:54:16.793079 (+6882427us) ts_tablet_manager.cc:1151] Deleting tablet b0b9e8eb767a4eceb583ff9b47000d58
0505 07:54:16.793090 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.805816 (+ 12726us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.820643 (+ 14827us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.822020 (+ 1377us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6882259}
I20260505 07:54:16.822476 2268 tablet_replica.cc:333] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.822688 2268 raft_consensus.cc:2243] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:16.822705 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:16.822971 2268 raft_consensus.cc:2272] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.823457 2268 ts_tablet_manager.cc:1916] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.823711 2276 ts_tablet_manager.cc:1916] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.824134 1358 catalog_manager.cc:5671] T 3462dcc6391741e59dcb5cd08527e50f P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:16.825033 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:16.825482 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:16.826252 1358 catalog_manager.cc:5671] T 37765950e67d466683551f67a35e9bdb P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:16.831898 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:16.832475 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:16.832938 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:16.833751 1358 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:16.838583 3013 consensus_queue.cc:1048] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:16.844363 2276 ts_tablet_manager.cc:1929] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.844487 2276 log.cc:1199] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/93d878ae0ddb42319560f74e0a490321
I20260505 07:54:16.844852 2276 ts_tablet_manager.cc:1950] T 93d878ae0ddb42319560f74e0a490321 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.845166 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 95) took 9072 ms. Trace:
I20260505 07:54:16.845248 2276 rpcz_store.cc:276] 0505 07:54:07.772798 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.772927 (+ 129us) service_pool.cc:224] Handling call
0505 07:54:16.818534 (+9045607us) ts_tablet_manager.cc:1151] Deleting tablet 93d878ae0ddb42319560f74e0a490321
0505 07:54:16.818547 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.826757 (+ 8210us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.844309 (+ 17552us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.845147 (+ 838us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9015814}
I20260505 07:54:16.845522 2276 tablet_replica.cc:333] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.845711 2276 raft_consensus.cc:2243] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.848842 2276 raft_consensus.cc:2272] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.849367 2276 ts_tablet_manager.cc:1916] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:16.851270 717 consensus_peers.cc:597] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ff7ed926a3ff4e7993864de734ca3c50. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:16.726765 856 leader_election.cc:343] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.853456 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbfb2345a1a4b218c2dddbae6667e47" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
W20260505 07:54:16.853672 854 leader_election.cc:343] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.853699 918 raft_consensus.cc:3060] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:16.853888 854 leader_election.cc:304] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:16.854074 854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 121213us
W20260505 07:54:16.854346 854 leader_election.cc:343] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.854486 2752 raft_consensus.cc:2749] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:16.854593 2752 raft_consensus.cc:493] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.854671 2752 raft_consensus.cc:515] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.855293 2752 leader_election.cc:290] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:16.856546 918 raft_consensus.cc:2468] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:16.856562 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f00169aad9420b892100acb4256f92" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:16.857050 854 leader_election.cc:343] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.857815 2268 ts_tablet_manager.cc:1929] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.857923 2268 log.cc:1199] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/d622612bdc404727a27b83769bf39738
I20260505 07:54:16.857971 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 21b48208e68247e78c12f0fec7ba07dc
I20260505 07:54:16.858243 2268 ts_tablet_manager.cc:1950] T d622612bdc404727a27b83769bf39738 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.858484 1350 catalog_manager.cc:6741] ReplaceTablet: tablet 21b48208e68247e78c12f0fec7ba07dc deleted and replaced by tablet 27ab0ebc13f349f38cf86b9643700ae1
I20260505 07:54:16.858587 1350 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 196) took 3427 ms. Trace:
I20260505 07:54:16.858960 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 186) took 6947 ms. Trace:
I20260505 07:54:16.858685 1350 rpcz_store.cc:276] 0505 07:54:13.431486 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.482770 (+3051284us) service_pool.cc:224] Handling call
0505 07:54:16.858559 (+375789us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.853445 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.853554 (+ 109us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.853559 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:16.853563 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:16.853699 (+ 136us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.853710 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.853717 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:16.853786 (+ 69us) tablet.cc:701] Row locks acquired
0505 07:54:16.853788 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:16.853834 (+ 46us) write_op.cc:270] Start()
0505 07:54:16.853858 (+ 24us) write_op.cc:276] Timestamp: P: 1777967656853743 usec, L: 0
0505 07:54:16.853863 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.854012 (+ 149us) log.cc:844] Serialized 622 byte log entry
0505 07:54:16.856852 (+ 2840us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.856944 (+ 92us) write_op.cc:301] APPLY: starting
0505 07:54:16.856956 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.856989 (+ 33us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.857018 (+ 29us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.857097 (+ 79us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.857138 (+ 41us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:16.857145 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:16.857197 (+ 52us) log.cc:844] Serialized 29 byte log entry
0505 07:54:16.857240 (+ 43us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.857254 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:16.857259 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.857310 (+ 51us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.857313 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":52,"apply.run_cpu_time_us":360,"apply.run_wall_time_us":360,"num_ops":2,"prepare.queue_time_us":39,"prepare.run_cpu_time_us":651,"prepare.run_wall_time_us":651,"raft.queue_time_us":53,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":210,"replication_time_us":2959}]]}
I20260505 07:54:16.859030 2268 rpcz_store.cc:276] 0505 07:54:09.911017 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.911257 (+ 240us) service_pool.cc:224] Handling call
0505 07:54:16.822438 (+6911181us) ts_tablet_manager.cc:1151] Deleting tablet d622612bdc404727a27b83769bf39738
0505 07:54:16.822447 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.846855 (+ 24408us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.857768 (+ 10913us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.858943 (+ 1175us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6911024}
I20260505 07:54:16.859269 2268 tablet_replica.cc:333] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.859362 1350 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_a44f4d613a6e47e38e9741252ef6956a" } new_table_name: "default.Table_6235f4345c354a6daa96d2ece798f82e" modify_external_catalogs: true
I20260505 07:54:16.859441 2268 raft_consensus.cc:2243] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.859575 2268 raft_consensus.cc:2272] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.860097 2268 ts_tablet_manager.cc:1916] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.860764 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 21b48208e68247e78c12f0fec7ba07dc with delete_type TABLET_DATA_DELETED (replaced by tablet 27ab0ebc13f349f38cf86b9643700ae1) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:16.868070 2276 ts_tablet_manager.cc:1929] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.868176 2276 log.cc:1199] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/b5a0cbf9c3d74cea92920189da332c1c
I20260505 07:54:16.868573 2276 ts_tablet_manager.cc:1950] T b5a0cbf9c3d74cea92920189da332c1c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.868882 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 96) took 9093 ms. Trace:
I20260505 07:54:16.868978 2276 rpcz_store.cc:276] 0505 07:54:07.775051 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.775282 (+ 231us) service_pool.cc:224] Handling call
0505 07:54:16.845480 (+9070198us) ts_tablet_manager.cc:1151] Deleting tablet b5a0cbf9c3d74cea92920189da332c1c
0505 07:54:16.845489 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.863427 (+ 17938us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.868020 (+ 4593us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.868864 (+ 844us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9042727}
I20260505 07:54:16.869284 2276 tablet_replica.cc:333] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.869496 2276 raft_consensus.cc:2243] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.869649 2276 raft_consensus.cc:2272] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:16.873617 420 external_mini_cluster.cc:1715] Unable to check leaks on /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu pid 1818: Timed out: CheckLeaks RPC to 127.0.105.60:35401 timed out after 1.000s (SENT)
I20260505 07:54:16.873811 420 external_mini_cluster.cc:1658] Killing /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu with pid 1818
W20260505 07:54:16.875383 991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260505 07:54:16.875598 991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260505 07:54:16.876020 3101 tablet.cc:1724] T 76939b7978ee4f7ea0ad29312de2a31a P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:16.877084 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet c1a61c1d03d544c8aa01537ead97f83c with delete_type TABLET_DATA_DELETED (replaced by tablet 080b2984fcf2478a8def6540da63554d) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.877427 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.877481 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 with delete_type TABLET_DATA_DELETED (replaced by tablet 6f207a3ffbc94b01abd9e9d6837f5c81) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.877776 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.877887 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 93d878ae0ddb42319560f74e0a490321 with delete_type TABLET_DATA_DELETED (replaced by tablet bc9ae6404dd64bd8bf4898768e94e4f1) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.878230 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.878612 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 05351965bdba4ccca8d13223cd505948 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.878943 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f3e076892234c1494b1a5c13cbfd2e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.879282 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet a9494c3b6a99487596a030856b371b68 with delete_type TABLET_DATA_DELETED (replaced by tablet 6fe19306f7df49fdb771d21f6560e697) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.879527 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 8b406c4c6ba741bc85eef54c6f576459 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.879827 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 1852f7f4ff594639908b03730b68fe86 with delete_type TABLET_DATA_DELETED (replaced by tablet aab3850799254dda9716fd5bf9404159) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.880136 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 02716f62aa354d8f839266661b7f05ea with delete_type TABLET_DATA_DELETED (replaced by tablet 9f541eff399641dd81a0771186805493) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.880447 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 09b46ff0101a4e529077b9e30492ace7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.880698 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.880980 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 397d33e908af4a0bb8dde26d273be01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:16.881446 3101 tablet.cc:1724] T 66082a7a75e345639e7a268727922fbb P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:16.882107 2650 raft_consensus.cc:493] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.882207 2650 raft_consensus.cc:515] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.882447 3013 consensus_queue.cc:1048] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:16.882812 2650 leader_election.cc:290] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:16.886077 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 159315us
W20260505 07:54:16.886432 856 leader_election.cc:343] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:16.886554 856 leader_election.cc:304] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.886752 2505 raft_consensus.cc:2749] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:16.886948 2650 raft_consensus.cc:493] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:16.887085 2650 raft_consensus.cc:515] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.887178 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet feeafd614b3e4281a8e45b950e699b63 (table default.Table_ad3e5b8653224a0c929e29e9daf47ff5 [id=99a81018fe1b45638089d579b76029d4]) successfully deleted
I20260505 07:54:16.888027 2650 leader_election.cc:290] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:16.893832 2268 ts_tablet_manager.cc:1929] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.895875 2276 ts_tablet_manager.cc:1916] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.903893 2268 log.cc:1199] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/3f1feb0387bc4afead72346398dbe9e5
I20260505 07:54:16.904336 2268 ts_tablet_manager.cc:1950] T 3f1feb0387bc4afead72346398dbe9e5 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.905118 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 187) took 6993 ms. Trace:
I20260505 07:54:16.905243 2268 rpcz_store.cc:276] 0505 07:54:09.911600 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.911902 (+ 302us) service_pool.cc:224] Handling call
0505 07:54:16.859230 (+6947328us) ts_tablet_manager.cc:1151] Deleting tablet 3f1feb0387bc4afead72346398dbe9e5
0505 07:54:16.859238 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.890758 (+ 31520us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.893793 (+ 3035us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.905095 (+ 11302us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6947199}
I20260505 07:54:16.905614 2268 tablet_replica.cc:333] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.905858 2268 raft_consensus.cc:2243] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.907330 2276 ts_tablet_manager.cc:1929] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.907536 2268 raft_consensus.cc:2272] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.910367 2276 log.cc:1199] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/397d33e908af4a0bb8dde26d273be01a
I20260505 07:54:16.911032 2268 ts_tablet_manager.cc:1916] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.917431 2268 ts_tablet_manager.cc:1929] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.917856 2276 ts_tablet_manager.cc:1950] T 397d33e908af4a0bb8dde26d273be01a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.918200 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 97) took 9142 ms. Trace:
I20260505 07:54:16.918318 2276 rpcz_store.cc:276] 0505 07:54:07.776018 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.776182 (+ 164us) service_pool.cc:224] Handling call
0505 07:54:16.869236 (+9093054us) ts_tablet_manager.cc:1151] Deleting tablet 397d33e908af4a0bb8dde26d273be01a
0505 07:54:16.869246 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.898526 (+ 29280us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.907287 (+ 8761us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.918186 (+ 10899us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9066443}
I20260505 07:54:16.918694 2276 tablet_replica.cc:333] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.919014 2276 raft_consensus.cc:2243] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.919216 2276 raft_consensus.cc:2272] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.920562 2276 ts_tablet_manager.cc:1916] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.797736 2447 rpcz_store.cc:276] 0505 07:54:09.552181 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.552237 (+ 56us) service_pool.cc:224] Handling call
0505 07:54:15.891062 (+6338825us) ts_tablet_manager.cc:1151] Deleting tablet 9a06f77255544d7dabb71a4f70ada809
0505 07:54:15.891070 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.362855 (+471785us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.566143 (+203288us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.797633 (+231490us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6332056}
I20260505 07:54:16.925359 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 9a06f77255544d7dabb71a4f70ada809 (table default.Table_8091d8a01cbf47f3b270874cd5e6140e [id=2a8cda02af654aa5b0c161e04968c735]) successfully deleted
I20260505 07:54:16.928848 1176 tablet_bootstrap.cc:654] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:16.917464 989 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260505 07:54:16.923336 988 leader_election.cc:304] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:16.933399 2948 raft_consensus.cc:2749] T ed0d43d85fcd4d05abf10d415e8dafbe P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:16.933744 2948 consensus_queue.cc:1048] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:16.934616 1181 tablet_bootstrap.cc:492] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.934790 1181 ts_tablet_manager.cc:1403] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.172s user 0.006s sys 0.019s
I20260505 07:54:16.934922 762 tablet_service.cc:1511] Processing CreateTablet for tablet 823cd847ae2b496ca7e99998297ba2fe (DEFAULT_TABLE table=default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.935406 1181 raft_consensus.cc:359] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.935552 1181 raft_consensus.cc:385] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.935621 1181 raft_consensus.cc:740] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.935796 1181 consensus_queue.cc:260] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.936287 1181 ts_tablet_manager.cc:1434] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260505 07:54:16.936579 1181 tablet_bootstrap.cc:492] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.936311 2268 log.cc:1199] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/03f31efbc98c484c950c70d5e84d2136
I20260505 07:54:16.937328 2268 ts_tablet_manager.cc:1950] T 03f31efbc98c484c950c70d5e84d2136 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.937964 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 188) took 7025 ms. Trace:
I20260505 07:54:16.938045 2268 rpcz_store.cc:276] 0505 07:54:09.912163 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.912456 (+ 293us) service_pool.cc:224] Handling call
0505 07:54:16.905566 (+6993110us) ts_tablet_manager.cc:1151] Deleting tablet 03f31efbc98c484c950c70d5e84d2136
0505 07:54:16.905577 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.914603 (+ 9026us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.917391 (+ 2788us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.937949 (+ 20558us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":154,"tablet-delete.queue_time_us":6992968}
I20260505 07:54:16.938288 2268 tablet_replica.cc:333] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.938446 2268 raft_consensus.cc:2243] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.938583 2268 raft_consensus.cc:2272] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.939028 2268 ts_tablet_manager.cc:1916] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.939435 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 0459f2d45a2d44f1a73e0a0fff53866c (table default.Table_01a5eabcd2ff41fca0d5c94e26775754 [id=0a65eba2d4074216964f45dbb0217d7a]) successfully deleted
I20260505 07:54:16.939850 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 93d878ae0ddb42319560f74e0a490321 (table default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]) successfully deleted
I20260505 07:54:16.940519 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet b5a0cbf9c3d74cea92920189da332c1c (table default.Table_e2720de78a7742d5829d7db2ef803f0d [id=f99b727e2aa54591baf6f109c07c2677]) successfully deleted
I20260505 07:54:16.942078 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 21b48208e68247e78c12f0fec7ba07dc with delete_type TABLET_DATA_DELETED (replaced by tablet 27ab0ebc13f349f38cf86b9643700ae1) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:16.942224 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 397d33e908af4a0bb8dde26d273be01a (table default.Table_499d356daab040e29ca2f5eb2941d790 [id=bf9eb924eb1e4586a6346bcf43be9a30]) successfully deleted
I20260505 07:54:16.942425 899 tablet_service.cc:1511] Processing CreateTablet for tablet 823cd847ae2b496ca7e99998297ba2fe (DEFAULT_TABLE table=default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:16.942646 2276 ts_tablet_manager.cc:1929] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:16.942723 2276 log.cc:1199] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e5eaa91ecc1f43d18b313b6be4cc211e
I20260505 07:54:16.942741 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 823cd847ae2b496ca7e99998297ba2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:16.942991 2276 ts_tablet_manager.cc:1950] T e5eaa91ecc1f43d18b313b6be4cc211e P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.943256 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 98) took 9166 ms. Trace:
I20260505 07:54:16.943316 2276 rpcz_store.cc:276] 0505 07:54:07.776577 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.776626 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:16.918651 (+9142025us) ts_tablet_manager.cc:1151] Deleting tablet e5eaa91ecc1f43d18b313b6be4cc211e
0505 07:54:16.918660 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.926002 (+ 7342us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.942601 (+ 16599us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.943243 (+ 642us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9115812}
I20260505 07:54:16.943553 2276 tablet_replica.cc:333] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.943742 2276 raft_consensus.cc:2243] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.943867 2276 raft_consensus.cc:2272] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.943766 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f00169aad9420b892100acb4256f92" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:16.944295 2276 ts_tablet_manager.cc:1916] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.951062 1181 tablet_bootstrap.cc:654] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.956420 2276 ts_tablet_manager.cc:1929] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.956530 2276 log.cc:1199] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/d8c9eaaef250461ca7a37d0cf42b07ee
I20260505 07:54:16.956882 2276 ts_tablet_manager.cc:1950] T d8c9eaaef250461ca7a37d0cf42b07ee P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:16.957159 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 99) took 9180 ms. Trace:
I20260505 07:54:16.957234 2276 rpcz_store.cc:276] 0505 07:54:07.776866 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.777728 (+ 862us) service_pool.cc:224] Handling call
0505 07:54:16.943520 (+9165792us) ts_tablet_manager.cc:1151] Deleting tablet d8c9eaaef250461ca7a37d0cf42b07ee
0505 07:54:16.943528 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.946946 (+ 3418us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.956378 (+ 9432us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.957144 (+ 766us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9140644}
I20260505 07:54:16.957517 2276 tablet_replica.cc:333] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:16.957690 2276 raft_consensus.cc:2243] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:16.958021 2276 raft_consensus.cc:2272] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.958580 2276 ts_tablet_manager.cc:1916] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.968051 1176 tablet_bootstrap.cc:492] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:16.968221 1176 ts_tablet_manager.cc:1403] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.174s user 0.002s sys 0.001s
I20260505 07:54:16.969007 1176 raft_consensus.cc:359] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.969206 1176 raft_consensus.cc:385] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.969295 1176 raft_consensus.cc:740] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.969516 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:16.969508 1176 consensus_queue.cc:260] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.969643 1172 tablet_bootstrap.cc:492] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:16.969762 1172 ts_tablet_manager.cc:1403] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.493s user 0.001s sys 0.002s
I20260505 07:54:16.969936 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:16.970099 1176 ts_tablet_manager.cc:1434] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:16.970247 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f106a6d4855148fe83b1e90ea9f27d39 (table default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00]). Expected version 1 got 0
I20260505 07:54:16.970482 2268 ts_tablet_manager.cc:1929] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.970585 2268 log.cc:1199] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/323269e708964848a94c64e6a781a860
I20260505 07:54:16.970600 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:16.970698 2752 raft_consensus.cc:493] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:16.970839 2752 raft_consensus.cc:515] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:16.970947 2268 ts_tablet_manager.cc:1950] T 323269e708964848a94c64e6a781a860 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.970963 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:16.971279 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:16.971503 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:16.971509 2752 leader_election.cc:290] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:16.971876 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:16.971911 1181 tablet_bootstrap.cc:492] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:16.972023 1181 ts_tablet_manager.cc:1403] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.036s user 0.003s sys 0.000s
I20260505 07:54:16.972033 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:16.972275 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:16.972486 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:16.972602 1181 raft_consensus.cc:359] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.972777 1181 raft_consensus.cc:385] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:16.972838 1181 raft_consensus.cc:740] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:16.972867 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:16.973006 1181 consensus_queue.cc:260] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:16.973254 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:16.973467 1181 ts_tablet_manager.cc:1434] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260505 07:54:16.973485 854 leader_election.cc:343] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: f88a5c0d0e2c436c8d8b6afc1f3c2edf
I20260505 07:54:16.973672 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:16.973744 1181 tablet_bootstrap.cc:492] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:16.973875 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:16.974176 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:16.974807 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 490483562d8f4c2daa7eaa333a9f3a0c (table default.Table_3acf7c3eb13d473a9d952098c7524753 [id=fe7ee6e8118840e8a9f70a007d80c2b3]). Expected version 1 got 0
I20260505 07:54:16.975133 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:16.975510 1181 tablet_bootstrap.cc:654] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:16.976601 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_c9015cf60d4c41b18743d55fbdcdf65e [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 1 got 0
I20260505 07:54:16.976827 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:16.976963 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:16.977151 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:16.977183 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e5eaa91ecc1f43d18b313b6be4cc211e (table default.Table_1ff5286907c6464c8addf47c70eef7ef [id=f608e93cbc64485297c6dc2609a6eb87]) successfully deleted
I20260505 07:54:16.977605 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:16.978037 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:16.978245 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:16.978466 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:16.978916 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet d8c9eaaef250461ca7a37d0cf42b07ee (table default.Table_92db3a6ebf824942966e677a2cd23ce1 [id=4890bd76a6f04392848038377244273d]) successfully deleted
I20260505 07:54:16.978973 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:16.979540 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
W20260505 07:54:16.980072 856 consensus_peers.cc:597] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 02e06d8815034d91986cef12964860ea. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:16.980139 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:16.980897 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:16.981318 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:16.981935 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:16.982543 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:16.982785 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:16.982932 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:16.983350 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:16.983615 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:16.984133 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:16.984164 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 189) took 7071 ms. Trace:
I20260505 07:54:16.984231 2268 rpcz_store.cc:276] 0505 07:54:09.912718 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.912977 (+ 259us) service_pool.cc:224] Handling call
0505 07:54:16.938252 (+7025275us) ts_tablet_manager.cc:1151] Deleting tablet 323269e708964848a94c64e6a781a860
0505 07:54:16.938260 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.963715 (+ 25455us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.970432 (+ 6717us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.984147 (+ 13715us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7025105}
I20260505 07:54:16.984371 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:16.984616 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:16.984862 1357 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
W20260505 07:54:16.985355 1353 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e38f248556bd4e49b06da72a1cbd7d8f consensus state has no leader
I20260505 07:54:16.985591 1353 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 197) took 3373 ms. Trace:
I20260505 07:54:16.984530 2268 tablet_replica.cc:333] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:16.986033 2268 raft_consensus.cc:2243] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:16.986178 2268 raft_consensus.cc:2272] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:16.985718 1353 rpcz_store.cc:276] 0505 07:54:13.612085 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.520470 (+2908385us) service_pool.cc:224] Handling call
0505 07:54:16.520860 (+ 390us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:16.521073 (+ 213us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:16.556289 (+ 35216us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:16.556378 (+ 89us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:16.566919 (+ 10541us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:16.985546 (+418627us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.556977 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.557098 (+ 121us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.557107 (+ 9us) write_op.cc:435] Acquired schema lock
0505 07:54:16.557111 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:16.557251 (+ 140us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.557269 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.557276 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:16.557351 (+ 75us) tablet.cc:701] Row locks acquired
0505 07:54:16.557354 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:16.557394 (+ 40us) write_op.cc:270] Start()
0505 07:54:16.557434 (+ 40us) write_op.cc:276] Timestamp: P: 1777967656557346 usec, L: 0
0505 07:54:16.557438 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.557666 (+ 228us) log.cc:844] Serialized 405 byte log entry
0505 07:54:16.566272 (+ 8606us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.566397 (+ 125us) write_op.cc:301] APPLY: starting
0505 07:54:16.566416 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.566468 (+ 52us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.566524 (+ 56us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.566586 (+ 62us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.566625 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:16.566645 (+ 20us) write_op.cc:312] APPLY: finished
0505 07:54:16.566733 (+ 88us) log.cc:844] Serialized 23 byte log entry
0505 07:54:16.566805 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.566826 (+ 21us) write_op.cc:454] Released schema lock
0505 07:54:16.566835 (+ 9us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.566909 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.566913 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":75,"apply.run_cpu_time_us":505,"apply.run_wall_time_us":504,"num_ops":1,"prepare.queue_time_us":54,"prepare.run_cpu_time_us":816,"prepare.run_wall_time_us":816,"raft.queue_time_us":1690,"raft.run_cpu_time_us":555,"raft.run_wall_time_us":554,"replication_time_us":8797}]]}
I20260505 07:54:16.986455 1353 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_fbe275b2392f4ab7a874b451ad147cb3" } modify_external_catalogs: true
I20260505 07:54:16.986647 2268 ts_tablet_manager.cc:1916] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:16.986647 1353 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_fbe275b2392f4ab7a874b451ad147cb3" } modify_external_catalogs: true
I20260505 07:54:16.987514 1350 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 204) took 2236 ms. Trace:
I20260505 07:54:16.987617 1350 rpcz_store.cc:276] 0505 07:54:14.750550 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.859212 (+2108662us) service_pool.cc:224] Handling call
0505 07:54:16.859518 (+ 306us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:16.859760 (+ 242us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:16.859940 (+ 180us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:16.860016 (+ 76us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:16.867713 (+ 7697us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:16.987468 (+119755us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.861466 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.861554 (+ 88us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.861558 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:16.861560 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:16.861625 (+ 65us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.861644 (+ 19us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.861648 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:16.861689 (+ 41us) tablet.cc:701] Row locks acquired
0505 07:54:16.861691 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:16.861714 (+ 23us) write_op.cc:270] Start()
0505 07:54:16.861735 (+ 21us) write_op.cc:276] Timestamp: P: 1777967656861641 usec, L: 0
0505 07:54:16.861738 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.861862 (+ 124us) log.cc:844] Serialized 405 byte log entry
0505 07:54:16.865760 (+ 3898us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.865847 (+ 87us) write_op.cc:301] APPLY: starting
0505 07:54:16.865857 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.865883 (+ 26us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.865910 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.865962 (+ 52us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.865986 (+ 24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:16.865992 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:16.866049 (+ 57us) log.cc:844] Serialized 23 byte log entry
0505 07:54:16.866090 (+ 41us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.866103 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:16.866108 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.866157 (+ 49us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.866161 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":54,"apply.run_cpu_time_us":304,"apply.run_wall_time_us":303,"num_ops":1,"prepare.queue_time_us":1058,"prepare.run_cpu_time_us":452,"prepare.run_wall_time_us":451,"replication_time_us":3999}]]}
I20260505 07:54:16.988327 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_246d9d987a3347cdb0f5b4e0f39bcbaf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:16.988866 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_246d9d987a3347cdb0f5b4e0f39bcbaf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:16.996804 1350 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 206) took 1923 ms. Trace:
I20260505 07:54:16.996896 1350 rpcz_store.cc:276] 0505 07:54:15.073627 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.988139 (+1914512us) service_pool.cc:224] Handling call
0505 07:54:16.988963 (+ 824us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:16.989376 (+ 413us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:16.996561 (+ 7185us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:16.996650 (+ 89us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:16.996749 (+ 99us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.989867 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.989963 (+ 96us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.989969 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:16.989972 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:16.990059 (+ 87us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.990072 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.990077 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:16.990159 (+ 82us) tablet.cc:701] Row locks acquired
0505 07:54:16.990161 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:16.990191 (+ 30us) write_op.cc:270] Start()
0505 07:54:16.990221 (+ 30us) write_op.cc:276] Timestamp: P: 1777967656990162 usec, L: 0
0505 07:54:16.990224 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.990378 (+ 154us) log.cc:844] Serialized 433 byte log entry
0505 07:54:16.993599 (+ 3221us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.993713 (+ 114us) write_op.cc:301] APPLY: starting
0505 07:54:16.993740 (+ 27us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.993782 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.993804 (+ 22us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.993906 (+ 102us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.993946 (+ 40us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:16.993953 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:16.994016 (+ 63us) log.cc:844] Serialized 29 byte log entry
0505 07:54:16.994068 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.994085 (+ 17us) write_op.cc:454] Released schema lock
0505 07:54:16.994090 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.994146 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.994150 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":71,"apply.run_cpu_time_us":427,"apply.run_wall_time_us":427,"num_ops":2,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":596,"prepare.run_wall_time_us":596,"raft.queue_time_us":355,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":221,"replication_time_us":3350}]]}
I20260505 07:54:16.997633 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_76f464c1c3b84d418ffffab788dc5ea5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:16.998108 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_76f464c1c3b84d418ffffab788dc5ea5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.005760 1350 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 207) took 1923 ms. Trace:
I20260505 07:54:17.005899 1350 rpcz_store.cc:276] 0505 07:54:15.082502 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.997443 (+1914941us) service_pool.cc:224] Handling call
0505 07:54:16.998208 (+ 765us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:16.998588 (+ 380us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:17.005445 (+ 6857us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.005451 (+ 6us) write_op.cc:454] Released schema lock
0505 07:54:17.005510 (+ 59us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:17.005618 (+ 108us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:17.005702 (+ 84us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.999507 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.999629 (+ 122us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.999634 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:16.999636 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:16.999715 (+ 79us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.999725 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.999730 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:16.999788 (+ 58us) tablet.cc:701] Row locks acquired
0505 07:54:16.999790 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:16.999815 (+ 25us) write_op.cc:270] Start()
0505 07:54:16.999838 (+ 23us) write_op.cc:276] Timestamp: P: 1777967656999786 usec, L: 0
0505 07:54:16.999842 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.999996 (+ 154us) log.cc:844] Serialized 433 byte log entry
0505 07:54:17.003326 (+ 3330us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.003413 (+ 87us) write_op.cc:301] APPLY: starting
0505 07:54:17.003424 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.003458 (+ 34us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.003486 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.003568 (+ 82us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.003612 (+ 44us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:17.003619 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:17.003683 (+ 64us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.003731 (+ 48us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.003746 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:17.003751 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":49,"apply.run_cpu_time_us":379,"apply.run_wall_time_us":379,"num_ops":2,"prepare.queue_time_us":509,"prepare.run_cpu_time_us":568,"prepare.run_wall_time_us":567,"raft.queue_time_us":51,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":172,"replication_time_us":3460}]]}
I20260505 07:54:17.007079 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_6125fe0ed26848bfbb9ed9d193944fa1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.011783 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6125fe0ed26848bfbb9ed9d193944fa1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.016963 1350 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 208) took 1782 ms. Trace:
I20260505 07:54:17.017097 1350 rpcz_store.cc:276] 0505 07:54:15.234361 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.006904 (+1772543us) service_pool.cc:224] Handling call
0505 07:54:17.011925 (+ 5021us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:17.012305 (+ 380us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:17.016682 (+ 4377us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.016689 (+ 7us) write_op.cc:454] Released schema lock
0505 07:54:17.016756 (+ 67us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:17.016846 (+ 90us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:17.016913 (+ 67us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.012720 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.012804 (+ 84us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.012809 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:17.012811 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.012877 (+ 66us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.012887 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.012891 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.012976 (+ 85us) tablet.cc:701] Row locks acquired
0505 07:54:17.012978 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.013003 (+ 25us) write_op.cc:270] Start()
0505 07:54:17.013025 (+ 22us) write_op.cc:276] Timestamp: P: 1777967657012973 usec, L: 0
0505 07:54:17.013028 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.013147 (+ 119us) log.cc:844] Serialized 433 byte log entry
0505 07:54:17.015820 (+ 2673us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.015908 (+ 88us) write_op.cc:301] APPLY: starting
0505 07:54:17.015920 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.015955 (+ 35us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.015983 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.016092 (+ 109us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.016121 (+ 29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:17.016128 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:17.016184 (+ 56us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.016240 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.016255 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:17.016260 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":54,"apply.run_cpu_time_us":392,"apply.run_wall_time_us":392,"num_ops":2,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":496,"prepare.run_wall_time_us":496,"raft.queue_time_us":47,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":178,"replication_time_us":2768}]]}
I20260505 07:54:17.017997 2268 ts_tablet_manager.cc:1929] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.018101 2268 log.cc:1199] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/9959be9f83c2444282d012b61eef49b5
I20260505 07:54:17.018435 2268 ts_tablet_manager.cc:1950] T 9959be9f83c2444282d012b61eef49b5 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.018605 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_1d013a93011845449eeb37dd0b8a25b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.019338 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1d013a93011845449eeb37dd0b8a25b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.022413 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 190) took 7109 ms. Trace:
I20260505 07:54:17.022518 2268 rpcz_store.cc:276] 0505 07:54:09.913354 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.916658 (+ 3304us) service_pool.cc:224] Handling call
0505 07:54:16.984471 (+7067813us) ts_tablet_manager.cc:1151] Deleting tablet 9959be9f83c2444282d012b61eef49b5
0505 07:54:16.984481 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.999186 (+ 14705us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.017938 (+ 18752us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.022391 (+ 4453us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7067659}
I20260505 07:54:17.022895 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 191) took 7105 ms. Trace:
I20260505 07:54:17.022982 2268 rpcz_store.cc:276] 0505 07:54:09.917041 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.918174 (+ 1133us) service_pool.cc:224] Handling call
0505 07:54:17.022787 (+7104613us) ts_tablet_manager.cc:1151] Deleting tablet 6da10815e0dc41a1a07b6786882fffd3
0505 07:54:17.022800 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.022878 (+ 78us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7100066}
I20260505 07:54:17.023252 2268 tablet_replica.cc:333] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.023458 2268 raft_consensus.cc:2243] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.023851 2268 raft_consensus.cc:2272] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.024353 2268 ts_tablet_manager.cc:1916] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.030433 2831 raft_consensus.cc:493] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.030611 2831 raft_consensus.cc:515] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.032166 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd2e2514c72419390c1bb9bd13f1fb3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:17.032708 717 leader_election.cc:343] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:17.033615 988 leader_election.cc:343] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: a9494c3b6a99487596a030856b371b68
I20260505 07:54:17.033731 988 leader_election.cc:304] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:17.033903 988 raft_consensus.cc:2686] T a9494c3b6a99487596a030856b371b68 P d63a3e535f8e4dd4a88e447253405b77: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260505 07:54:17.038470 2831 leader_election.cc:290] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:17.019531 1349 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 212e496d564a4b5ebabb22504be76bf2
I20260505 07:54:17.039085 1349 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 200) took 3348 ms. Trace:
I20260505 07:54:17.039166 1349 rpcz_store.cc:276] 0505 07:54:13.690124 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.680621 (+2990497us) service_pool.cc:224] Handling call
0505 07:54:16.683517 (+ 2896us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:16.685327 (+ 1810us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:16.685383 (+ 56us) catalog_manager.cc:2824] Locking tablets
0505 07:54:16.915615 (+230232us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:16.920789 (+ 5174us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:16.920871 (+ 82us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:17.019492 (+ 98621us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:17.039036 (+ 19544us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.916276 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.916391 (+ 115us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.916398 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:16.916400 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:16.916509 (+ 109us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.916539 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.916544 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:16.916628 (+ 84us) tablet.cc:701] Row locks acquired
0505 07:54:16.916630 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:16.916672 (+ 42us) write_op.cc:270] Start()
0505 07:54:16.916699 (+ 27us) write_op.cc:276] Timestamp: P: 1777967656916639 usec, L: 0
0505 07:54:16.916703 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.916854 (+ 151us) log.cc:844] Serialized 725 byte log entry
0505 07:54:16.919975 (+ 3121us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.920060 (+ 85us) write_op.cc:301] APPLY: starting
0505 07:54:16.920084 (+ 24us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.920120 (+ 36us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.920148 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.920198 (+ 50us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.920226 (+ 28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:16.920232 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:16.920288 (+ 56us) log.cc:844] Serialized 29 byte log entry
0505 07:54:16.920334 (+ 46us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.920349 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:16.920355 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.920427 (+ 72us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.920431 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":48,"apply.run_cpu_time_us":344,"apply.run_wall_time_us":344,"num_ops":2,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":658,"prepare.run_wall_time_us":657,"raft.queue_time_us":55,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":175,"replication_time_us":3248}]]}
I20260505 07:54:17.039939 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_3783f0e113d542d9ad9b5c85f83624bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.040413 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3783f0e113d542d9ad9b5c85f83624bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:17.019851 1351 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1506cd31fe364e328867bbf70e36845e consensus state has no leader
I20260505 07:54:17.040673 1351 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 203) took 2290 ms. Trace:
I20260505 07:54:17.040757 1351 rpcz_store.cc:276] 0505 07:54:14.750437 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.724889 (+1974452us) service_pool.cc:224] Handling call
0505 07:54:16.725665 (+ 776us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:16.725873 (+ 208us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:16.726071 (+ 198us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:16.726143 (+ 72us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:16.738079 (+ 11936us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.738085 (+ 6us) write_op.cc:454] Released schema lock
0505 07:54:16.738144 (+ 59us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:17.040636 (+302492us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.731339 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.731430 (+ 91us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.731435 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:16.731437 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:16.731515 (+ 78us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.731527 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.731531 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:16.731580 (+ 49us) tablet.cc:701] Row locks acquired
0505 07:54:16.731582 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:16.731630 (+ 48us) write_op.cc:270] Start()
0505 07:54:16.731679 (+ 49us) write_op.cc:276] Timestamp: P: 1777967656731565 usec, L: 0
0505 07:54:16.731685 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.731874 (+ 189us) log.cc:844] Serialized 405 byte log entry
0505 07:54:16.735777 (+ 3903us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.735867 (+ 90us) write_op.cc:301] APPLY: starting
0505 07:54:16.735879 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.735913 (+ 34us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.735968 (+ 55us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.736010 (+ 42us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.736038 (+ 28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:16.736045 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:16.736106 (+ 61us) log.cc:844] Serialized 23 byte log entry
0505 07:54:16.736154 (+ 48us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.736169 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:16.736175 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":48,"apply.run_cpu_time_us":352,"apply.run_wall_time_us":351,"num_ops":1,"prepare.queue_time_us":4826,"prepare.run_cpu_time_us":639,"prepare.run_wall_time_us":640,"raft.queue_time_us":1290,"raft.run_cpu_time_us":477,"raft.run_wall_time_us":476,"replication_time_us":4057}]]}
I20260505 07:54:17.041383 1351 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_7b5084730d534072a607d530beced51c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:17.020031 1352 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 66f069982138460da3b7ae7a0b4e3833
I20260505 07:54:17.043663 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 212e496d564a4b5ebabb22504be76bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.043769 1352 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 198) took 3415 ms. Trace:
I20260505 07:54:17.043855 1352 rpcz_store.cc:276] 0505 07:54:13.628039 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.624930 (+2996891us) service_pool.cc:224] Handling call
0505 07:54:16.625533 (+ 603us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:16.625688 (+ 155us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:16.625744 (+ 56us) catalog_manager.cc:2824] Locking tablets
0505 07:54:16.625950 (+ 206us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:16.648344 (+ 22394us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:16.648436 (+ 92us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:17.020002 (+371566us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:17.043722 (+ 23720us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.637156 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.637259 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.637266 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:16.637269 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:16.637382 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.637397 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.637401 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:16.637501 (+ 100us) tablet.cc:701] Row locks acquired
0505 07:54:16.637504 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:16.637537 (+ 33us) write_op.cc:270] Start()
0505 07:54:16.637569 (+ 32us) write_op.cc:276] Timestamp: P: 1777967656637485 usec, L: 0
0505 07:54:16.637573 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.637761 (+ 188us) log.cc:844] Serialized 725 byte log entry
0505 07:54:16.643498 (+ 5737us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.643608 (+ 110us) write_op.cc:301] APPLY: starting
0505 07:54:16.643623 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.643677 (+ 54us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.643708 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.643788 (+ 80us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.643822 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:16.643829 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:16.643900 (+ 71us) log.cc:844] Serialized 29 byte log entry
0505 07:54:16.643959 (+ 59us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.643978 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:16.643984 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.644046 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.644049 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":50,"apply.run_cpu_time_us":433,"apply.run_wall_time_us":433,"num_ops":2,"prepare.queue_time_us":1001,"prepare.run_cpu_time_us":721,"prepare.run_wall_time_us":720,"raft.queue_time_us":964,"raft.run_cpu_time_us":671,"raft.run_wall_time_us":3979,"replication_time_us":5897}]]}
I20260505 07:54:17.045084 1352 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_45058c18f11e47608bcaad243991f75b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.045499 1352 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_45058c18f11e47608bcaad243991f75b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.045574 2268 ts_tablet_manager.cc:1929] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.033114 1355 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 5191f9a357de4607aab6c7a8b4bcccdb
I20260505 07:54:17.045661 2268 log.cc:1199] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/8a60c015136246a3b02f70254c0bb67f
I20260505 07:54:17.046046 2268 ts_tablet_manager.cc:1950] T 8a60c015136246a3b02f70254c0bb67f P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.046075 1355 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 199) took 3360 ms. Trace:
I20260505 07:54:17.046173 1355 rpcz_store.cc:276] 0505 07:54:13.685382 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.657532 (+2972150us) service_pool.cc:224] Handling call
0505 07:54:16.674482 (+ 16950us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:16.674629 (+ 147us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:16.674693 (+ 64us) catalog_manager.cc:2824] Locking tablets
0505 07:54:16.871784 (+197091us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:16.876877 (+ 5093us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:16.876954 (+ 77us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:17.033060 (+156106us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:17.046043 (+ 12983us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.872386 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.872479 (+ 93us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.872484 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:16.872486 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:16.872566 (+ 80us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.872576 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.872580 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:16.872667 (+ 87us) tablet.cc:701] Row locks acquired
0505 07:54:16.872669 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:16.872694 (+ 25us) write_op.cc:270] Start()
0505 07:54:16.872716 (+ 22us) write_op.cc:276] Timestamp: P: 1777967656872678 usec, L: 0
0505 07:54:16.872719 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.872838 (+ 119us) log.cc:844] Serialized 759 byte log entry
0505 07:54:16.876114 (+ 3276us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.876207 (+ 93us) write_op.cc:301] APPLY: starting
0505 07:54:16.876218 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.876246 (+ 28us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.876265 (+ 19us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.876315 (+ 50us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.876338 (+ 23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:16.876345 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:16.876395 (+ 50us) log.cc:844] Serialized 29 byte log entry
0505 07:54:16.876439 (+ 44us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.876454 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:16.876461 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:16.876523 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.876529 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":57,"apply.run_cpu_time_us":305,"apply.run_wall_time_us":305,"num_ops":2,"prepare.queue_time_us":40,"prepare.run_cpu_time_us":524,"prepare.run_wall_time_us":525,"raft.queue_time_us":275,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":166,"replication_time_us":3373}]]}
I20260505 07:54:17.046895 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_9e7869c6af694bebae3920cd0afa7bbc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.047292 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e7869c6af694bebae3920cd0afa7bbc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:17.034662 1348 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 68e608ac01ef4a88906e585bd7ee25ac consensus state has no leader
W20260505 07:54:17.047923 1351 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7b5084730d534072a607d530beced51c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.048138 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 66f069982138460da3b7ae7a0b4e3833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.048524 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 5191f9a357de4607aab6c7a8b4bcccdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.051854 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 193) took 7133 ms. Trace:
I20260505 07:54:17.051962 2268 rpcz_store.cc:276] 0505 07:54:09.918548 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.918595 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:17.023199 (+7104604us) ts_tablet_manager.cc:1151] Deleting tablet 8a60c015136246a3b02f70254c0bb67f
0505 07:54:17.023209 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.027492 (+ 4283us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.045529 (+ 18037us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.051831 (+ 6302us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7100412}
I20260505 07:54:17.052273 2268 tablet_replica.cc:333] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.052469 2268 raft_consensus.cc:2243] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.052621 2268 raft_consensus.cc:2272] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.053200 2268 ts_tablet_manager.cc:1916] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.054847 1348 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 202) took 3194 ms. Trace:
I20260505 07:54:17.054944 1348 rpcz_store.cc:276] 0505 07:54:13.860812 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.705799 (+2844987us) service_pool.cc:224] Handling call
0505 07:54:16.706782 (+ 983us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:16.706972 (+ 190us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:16.707115 (+ 143us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:16.707167 (+ 52us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:16.720888 (+ 13721us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.720897 (+ 9us) write_op.cc:454] Released schema lock
0505 07:54:16.720976 (+ 79us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:17.054814 (+333838us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.712267 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.712362 (+ 95us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.712367 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:16.712369 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:16.712452 (+ 83us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.712465 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.712471 (+ 6us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:16.712516 (+ 45us) tablet.cc:701] Row locks acquired
0505 07:54:16.712519 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:16.712543 (+ 24us) write_op.cc:270] Start()
0505 07:54:16.712569 (+ 26us) write_op.cc:276] Timestamp: P: 1777967656712485 usec, L: 0
0505 07:54:16.712572 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.712713 (+ 141us) log.cc:844] Serialized 405 byte log entry
0505 07:54:16.716778 (+ 4065us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.716857 (+ 79us) write_op.cc:301] APPLY: starting
0505 07:54:16.716868 (+ 11us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.716898 (+ 30us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.716952 (+ 54us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.716992 (+ 40us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.717034 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:16.717040 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:16.717101 (+ 61us) log.cc:844] Serialized 23 byte log entry
0505 07:54:16.717144 (+ 43us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.717159 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:16.717164 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":7279,"child_traces":[["op",{"apply.queue_time_us":43,"apply.run_cpu_time_us":347,"apply.run_wall_time_us":347,"num_ops":1,"prepare.queue_time_us":4690,"prepare.run_cpu_time_us":527,"prepare.run_wall_time_us":527,"raft.queue_time_us":336,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":368,"replication_time_us":4182}]]}
I20260505 07:54:17.055676 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_ba25fa8c94264de0bab2e95e8d9571ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.056119 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ba25fa8c94264de0bab2e95e8d9571ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:17.058319 988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260505 07:54:16.980425 2276 ts_tablet_manager.cc:1929] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:16.999033 2505 raft_consensus.cc:493] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.059758 2276 log.cc:1199] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/62d4bc3154704ba2bfde83bca10f812d
I20260505 07:54:17.059795 2505 raft_consensus.cc:515] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.060134 2276 ts_tablet_manager.cc:1950] T 62d4bc3154704ba2bfde83bca10f812d P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.060415 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 100) took 9282 ms. Trace:
I20260505 07:54:17.060513 2276 rpcz_store.cc:276] 0505 07:54:07.777974 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.778021 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:16.957478 (+9179457us) ts_tablet_manager.cc:1151] Deleting tablet 62d4bc3154704ba2bfde83bca10f812d
0505 07:54:16.957487 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:16.977585 (+ 20098us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:16.980382 (+ 2797us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.060401 (+ 80019us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9154559}
I20260505 07:54:17.060627 2505 leader_election.cc:290] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:17.060782 2276 tablet_replica.cc:333] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.060972 2276 raft_consensus.cc:2243] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.061111 2276 raft_consensus.cc:2272] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.061625 2276 ts_tablet_manager.cc:1916] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:17.063560 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 83523.7us
I20260505 07:54:17.063623 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f5b72f58a24db0b58c15b47227bc60" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:17.063769 856 leader_election.cc:343] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:17.068161 1348 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 214) took 1597 ms. Trace:
I20260505 07:54:17.068267 1348 rpcz_store.cc:276] 0505 07:54:15.470745 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.055487 (+1584742us) service_pool.cc:224] Handling call
0505 07:54:17.056198 (+ 711us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:17.056588 (+ 390us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:17.067940 (+ 11352us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:17.068032 (+ 92us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:17.068114 (+ 82us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.059720 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.059816 (+ 96us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.059822 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:17.059824 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.059944 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.059962 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.059969 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.060056 (+ 87us) tablet.cc:701] Row locks acquired
0505 07:54:17.060060 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:17.060099 (+ 39us) write_op.cc:270] Start()
0505 07:54:17.060133 (+ 34us) write_op.cc:276] Timestamp: P: 1777967657060061 usec, L: 0
0505 07:54:17.060137 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.060321 (+ 184us) log.cc:844] Serialized 433 byte log entry
0505 07:54:17.064924 (+ 4603us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.065069 (+ 145us) write_op.cc:301] APPLY: starting
0505 07:54:17.065088 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.065173 (+ 85us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.065205 (+ 32us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.065399 (+ 194us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.065450 (+ 51us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:17.065461 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:17.065550 (+ 89us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.065612 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.065633 (+ 21us) write_op.cc:454] Released schema lock
0505 07:54:17.065641 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.065746 (+ 105us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.065752 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":85,"apply.run_cpu_time_us":654,"apply.run_wall_time_us":653,"num_ops":2,"prepare.queue_time_us":2651,"prepare.run_cpu_time_us":702,"prepare.run_wall_time_us":702,"raft.queue_time_us":76,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":231,"replication_time_us":4739}]]}
I20260505 07:54:17.068989 1348 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_48bc29e8382d4b5ba99601d826206b08"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.069480 1348 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_48bc29e8382d4b5ba99601d826206b08 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.071168 2268 ts_tablet_manager.cc:1929] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.071257 2268 log.cc:1199] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/9044763bed1a41739ecce4def4cca1fe
I20260505 07:54:17.073894 1349 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 210) took 1788 ms. Trace:
I20260505 07:54:17.074021 1349 rpcz_store.cc:276] 0505 07:54:15.285772 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.039752 (+1753980us) service_pool.cc:224] Handling call
0505 07:54:17.040492 (+ 740us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:17.064293 (+ 23801us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:17.073585 (+ 9292us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:17.073704 (+ 119us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:17.073800 (+ 96us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.064835 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.064921 (+ 86us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.064926 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:17.064929 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:17.064997 (+ 68us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.065006 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.065010 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.065070 (+ 60us) tablet.cc:701] Row locks acquired
0505 07:54:17.065072 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.065098 (+ 26us) write_op.cc:270] Start()
0505 07:54:17.065123 (+ 25us) write_op.cc:276] Timestamp: P: 1777967657065062 usec, L: 0
0505 07:54:17.065126 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.065253 (+ 127us) log.cc:844] Serialized 433 byte log entry
0505 07:54:17.072604 (+ 7351us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.072733 (+ 129us) write_op.cc:301] APPLY: starting
0505 07:54:17.072749 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.072794 (+ 45us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.072836 (+ 42us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.072994 (+ 158us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.073034 (+ 40us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:17.073045 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:17.073122 (+ 77us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.073176 (+ 54us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.073195 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:17.073203 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.073283 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.073289 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":77,"apply.run_cpu_time_us":539,"apply.run_wall_time_us":538,"num_ops":2,"prepare.queue_time_us":48,"prepare.run_cpu_time_us":480,"prepare.run_wall_time_us":480,"replication_time_us":7445}]]}
I20260505 07:54:17.074971 1349 master_service.cc:954] ReplaceTablet: received request to replace tablet 9c4fc18f6faa488882d9bffb30793cdd from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:17.071573 2268 ts_tablet_manager.cc:1950] T 9044763bed1a41739ecce4def4cca1fe P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.076438 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 194) took 7157 ms. Trace:
I20260505 07:54:17.076541 2268 rpcz_store.cc:276] 0505 07:54:09.918840 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.918889 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:17.052220 (+7133331us) ts_tablet_manager.cc:1151] Deleting tablet 9044763bed1a41739ecce4def4cca1fe
0505 07:54:17.052232 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.068194 (+ 15962us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.071126 (+ 2932us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.076415 (+ 5289us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7129353}
I20260505 07:54:17.076866 2268 tablet_replica.cc:333] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.077165 2268 raft_consensus.cc:2243] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.077435 2268 raft_consensus.cc:2272] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.077970 2268 ts_tablet_manager.cc:1916] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.079943 1352 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 212) took 1673 ms. Trace:
I20260505 07:54:17.080029 1352 rpcz_store.cc:276] 0505 07:54:15.406841 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.044926 (+1638085us) service_pool.cc:224] Handling call
0505 07:54:17.045579 (+ 653us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:17.066037 (+ 20458us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:17.079741 (+ 13704us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:17.079827 (+ 86us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:17.079895 (+ 68us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.071639 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.071730 (+ 91us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.071735 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:17.071737 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.071799 (+ 62us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.071807 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.071811 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.071864 (+ 53us) tablet.cc:701] Row locks acquired
0505 07:54:17.071866 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.071889 (+ 23us) write_op.cc:270] Start()
0505 07:54:17.071911 (+ 22us) write_op.cc:276] Timestamp: P: 1777967657071855 usec, L: 0
0505 07:54:17.071914 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.072053 (+ 139us) log.cc:844] Serialized 433 byte log entry
0505 07:54:17.078850 (+ 6797us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.079148 (+ 298us) write_op.cc:301] APPLY: starting
0505 07:54:17.079166 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.079208 (+ 42us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.079239 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.079385 (+ 146us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.079425 (+ 40us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:17.079437 (+ 12us) write_op.cc:312] APPLY: finished
0505 07:54:17.079510 (+ 73us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.079645 (+ 135us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.079664 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:17.079671 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.079752 (+ 81us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.079758 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":242,"apply.run_cpu_time_us":566,"apply.run_wall_time_us":593,"num_ops":2,"prepare.queue_time_us":5191,"prepare.run_cpu_time_us":482,"prepare.run_wall_time_us":482,"replication_time_us":6905}]]}
I20260505 07:54:17.080603 1352 master_service.cc:954] ReplaceTablet: received request to replace tablet 8cf3c334ebb24956b50dbefc23d4d60b from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:17.086872 1351 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 211) took 1691 ms. Trace:
I20260505 07:54:17.087034 1351 rpcz_store.cc:276] 0505 07:54:15.395755 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.041247 (+1645492us) service_pool.cc:224] Handling call
0505 07:54:17.048004 (+ 6757us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:17.067374 (+ 19370us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:17.086631 (+ 19257us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:17.086738 (+ 107us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:17.086829 (+ 91us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.072601 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.072710 (+ 109us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.072714 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:17.072717 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:17.072780 (+ 63us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.072787 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.072791 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.072840 (+ 49us) tablet.cc:701] Row locks acquired
0505 07:54:17.072842 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.072863 (+ 21us) write_op.cc:270] Start()
0505 07:54:17.072882 (+ 19us) write_op.cc:276] Timestamp: P: 1777967657072829 usec, L: 0
0505 07:54:17.072885 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.073010 (+ 125us) log.cc:844] Serialized 433 byte log entry
0505 07:54:17.085163 (+ 12153us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.086163 (+ 1000us) write_op.cc:301] APPLY: starting
0505 07:54:17.086175 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.086209 (+ 34us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.086243 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.086345 (+ 102us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.086379 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:17.086389 (+ 10us) write_op.cc:312] APPLY: finished
0505 07:54:17.086446 (+ 57us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.086488 (+ 42us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.086503 (+ 15us) write_op.cc:454] Released schema lock
0505 07:54:17.086510 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":966,"apply.run_cpu_time_us":406,"apply.run_wall_time_us":405,"num_ops":2,"prepare.queue_time_us":4852,"prepare.run_cpu_time_us":448,"prepare.run_wall_time_us":447,"replication_time_us":12266,"threads_started":1}]]}
I20260505 07:54:17.088083 1355 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 213) took 1634 ms. Trace:
I20260505 07:54:17.088183 1355 rpcz_store.cc:276] 0505 07:54:15.453480 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.046752 (+1593272us) service_pool.cc:224] Handling call
0505 07:54:17.047371 (+ 619us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:17.066731 (+ 19360us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:17.087898 (+ 21167us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:17.087978 (+ 80us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:17.088044 (+ 66us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.072140 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.072239 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.072246 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:17.072248 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.072314 (+ 66us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.072321 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.072325 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.072377 (+ 52us) tablet.cc:701] Row locks acquired
0505 07:54:17.072379 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.072401 (+ 22us) write_op.cc:270] Start()
0505 07:54:17.072422 (+ 21us) write_op.cc:276] Timestamp: P: 1777967657072367 usec, L: 0
0505 07:54:17.072425 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.072534 (+ 109us) log.cc:844] Serialized 433 byte log entry
0505 07:54:17.085117 (+ 12583us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.085534 (+ 417us) write_op.cc:301] APPLY: starting
0505 07:54:17.085551 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.085594 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.085636 (+ 42us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.085765 (+ 129us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.085804 (+ 39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:17.085815 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:17.085923 (+ 108us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.085988 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.086008 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:17.086015 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.086088 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.086094 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":368,"apply.run_cpu_time_us":545,"apply.run_wall_time_us":545,"num_ops":2,"prepare.queue_time_us":5113,"prepare.run_cpu_time_us":434,"prepare.run_wall_time_us":433,"replication_time_us":12670}]]}
I20260505 07:54:17.096318 2268 ts_tablet_manager.cc:1929] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.096454 2268 log.cc:1199] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/06904a54b1c5463da31af0bc00fb606f
I20260505 07:54:17.096832 2268 ts_tablet_manager.cc:1950] T 06904a54b1c5463da31af0bc00fb606f P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:16.970470 1172 raft_consensus.cc:359] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.097579 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 196) took 7177 ms. Trace:
I20260505 07:54:17.097659 2268 rpcz_store.cc:276] 0505 07:54:09.920053 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.924248 (+ 4195us) service_pool.cc:224] Handling call
0505 07:54:17.076825 (+7152577us) ts_tablet_manager.cc:1151] Deleting tablet 06904a54b1c5463da31af0bc00fb606f
0505 07:54:17.076835 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.085309 (+ 8474us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.096268 (+ 10959us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.097560 (+ 1292us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7147082}
I20260505 07:54:17.098031 2268 tablet_replica.cc:333] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.098266 2268 raft_consensus.cc:2243] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.098486 2268 raft_consensus.cc:2272] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.099066 2268 ts_tablet_manager.cc:1916] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:17.101624 988 leader_election.cc:343] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Not found: Tablet not found: 003cd4472b424aacbba8c5f89900e228
I20260505 07:54:17.101766 988 leader_election.cc:304] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:17.101997 988 raft_consensus.cc:2686] T 003cd4472b424aacbba8c5f89900e228 P d63a3e535f8e4dd4a88e447253405b77: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260505 07:54:17.106140 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:17.115149 2268 ts_tablet_manager.cc:1929] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.115234 2268 log.cc:1199] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/26f3544e74d941d286a2a0317b941ab6
I20260505 07:54:17.115576 2268 ts_tablet_manager.cc:1950] T 26f3544e74d941d286a2a0317b941ab6 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.115903 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 197) took 7191 ms. Trace:
I20260505 07:54:17.116003 2268 rpcz_store.cc:276] 0505 07:54:09.924499 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.924544 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:17.097974 (+7173430us) ts_tablet_manager.cc:1151] Deleting tablet 26f3544e74d941d286a2a0317b941ab6
0505 07:54:17.097988 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.112162 (+ 14174us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.115109 (+ 2947us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.115888 (+ 779us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7168175}
I20260505 07:54:17.116266 2268 tablet_replica.cc:333] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.116451 2268 raft_consensus.cc:2243] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.116698 2268 raft_consensus.cc:2272] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.117188 2268 ts_tablet_manager.cc:1916] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.124470 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet d77ff90ec2ff4b6eb1795d149459c23a with delete_type TABLET_DATA_DELETED (replaced by tablet e5eaa91ecc1f43d18b313b6be4cc211e) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.125128 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 60772a9697ba417b879dc6025eeb8eec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.125509 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 02e06d8815034d91986cef12964860ea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.125905 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ef4f516e64a44fd8cd33674fb808067 with delete_type TABLET_DATA_DELETED (replaced by tablet 42360f8ee4ff47c2af8d287f792207de) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.126255 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a06f77255544d7dabb71a4f70ada809 with delete_type TABLET_DATA_DELETED (replaced by tablet 12841efcef6041d89fc6d7d9b90448c0) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.126586 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet cac5064f68d24ebe95b5a0b8e10f463e with delete_type TABLET_DATA_DELETED (replaced by tablet dd3fc8a9d9f04636b1f31bb88411c1da) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.126896 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet f1058fcb9fa6457a85489049ca1d9ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.127357 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet bd581a6799a14a8dafb80d411e8936f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.127707 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 7e707143a4e64725aa66db05b24999a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.128043 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 909306e1e5f34b7c8fbc0d86cafce531 with delete_type TABLET_DATA_DELETED (replaced by tablet 66082a7a75e345639e7a268727922fbb) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.128377 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet e82e9ae417784b7e83f4843b129d0580 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.063854 856 leader_election.cc:304] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:17.130870 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
W20260505 07:54:17.064092 854 leader_election.cc:343] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:17.131760 854 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 67671.1us
I20260505 07:54:17.132388 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2706bc9b9e02414693062deff5cfe81b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.132607 920 raft_consensus.cc:2393] T 2706bc9b9e02414693062deff5cfe81b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:17.135902 2268 ts_tablet_manager.cc:1929] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.135999 2268 log.cc:1199] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/bf76e868bbcb44b0b1887c84dd27f093
I20260505 07:54:17.136361 2268 ts_tablet_manager.cc:1950] T bf76e868bbcb44b0b1887c84dd27f093 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.140954 2447 tablet_replica.cc:333] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:17.141207 2447 raft_consensus.cc:2243] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.141410 2447 raft_consensus.cc:2272] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:17.141454 717 consensus_peers.cc:597] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f1feb0387bc4afead72346398dbe9e5. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:17.067267 2276 ts_tablet_manager.cc:1929] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:17.141772 2276 log.cc:1199] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/8b57c1c618484d5d86a846fb982a2240
I20260505 07:54:17.142134 2276 ts_tablet_manager.cc:1950] T 8b57c1c618484d5d86a846fb982a2240 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.142355 2447 ts_tablet_manager.cc:1916] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.142447 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 101) took 9364 ms. Trace:
I20260505 07:54:17.142527 2276 rpcz_store.cc:276] 0505 07:54:07.778244 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.778290 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:17.060740 (+9282450us) ts_tablet_manager.cc:1151] Deleting tablet 8b57c1c618484d5d86a846fb982a2240
0505 07:54:17.060749 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.064603 (+ 3854us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.067227 (+ 2624us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.142432 (+ 75205us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9257772}
I20260505 07:54:17.142817 2276 tablet_replica.cc:333] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.097738 1172 raft_consensus.cc:385] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.143082 1172 raft_consensus.cc:740] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.143829 1172 consensus_queue.cc:260] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.146140 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 823cd847ae2b496ca7e99998297ba2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.146194 1176 tablet_bootstrap.cc:492] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:17.147564 1176 tablet_bootstrap.cc:654] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.154146 3100 tablet.cc:1724] T 429ccdb56f0b456483cf1debb5de0beb P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:17.157318 988 leader_election.cc:304] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
W20260505 07:54:17.157532 988 raft_consensus.cc:2686] T 27bacc97d7534284970f218cf68ac75b P d63a3e535f8e4dd4a88e447253405b77: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260505 07:54:17.160908 1315 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_65d14038399640c284b8382aa80e344f [id=fc9aba6852ee42cc83697f86e0e69c5e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 68e608ac01ef4a88906e585bd7ee25ac consensus state has no leader
I20260505 07:54:17.162532 3112 tablet.cc:1724] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:17.168756 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4515a0ff8304c69b9aa030e9b14d43f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.169052 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4515a0ff8304c69b9aa030e9b14d43f" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:17.168953 907 raft_consensus.cc:2468] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:17.171535 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abaa596871bb4a82aefc317932e91d35" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.171765 921 raft_consensus.cc:2393] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
W20260505 07:54:17.175841 717 consensus_peers.cc:597] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0baf83b155494fddb3c25144608706f7. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:17.182830 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 199) took 7257 ms. Trace:
I20260505 07:54:17.182927 2268 rpcz_store.cc:276] 0505 07:54:09.925166 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.925211 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:17.116229 (+7191018us) ts_tablet_manager.cc:1151] Deleting tablet bf76e868bbcb44b0b1887c84dd27f093
0505 07:54:17.116238 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.133105 (+ 16867us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.135849 (+ 2744us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.182809 (+ 46960us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7186385}
I20260505 07:54:17.183229 2268 tablet_replica.cc:333] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.183408 2268 raft_consensus.cc:2243] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.183544 2268 raft_consensus.cc:2272] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.184088 2268 ts_tablet_manager.cc:1916] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.184464 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet ad5c570d4861460d8ab0c8ff8a7a3a03 with delete_type TABLET_DATA_DELETED (replaced by tablet b53db8fdadad4ec09d05f65c84100ff0) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.184896 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 21b48208e68247e78c12f0fec7ba07dc with delete_type TABLET_DATA_DELETED (replaced by tablet 27ab0ebc13f349f38cf86b9643700ae1) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.185442 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 823cd847ae2b496ca7e99998297ba2fe (DEFAULT_TABLE table=default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:17.185863 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 823cd847ae2b496ca7e99998297ba2fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.189976 2268 ts_tablet_manager.cc:1929] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.190058 2268 log.cc:1199] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/23fa401bb4634022bca3c6b23421e4d5
I20260505 07:54:17.190384 2268 ts_tablet_manager.cc:1950] T 23fa401bb4634022bca3c6b23421e4d5 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.190994 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 200) took 7265 ms. Trace:
I20260505 07:54:17.191064 2268 rpcz_store.cc:276] 0505 07:54:09.925493 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.925886 (+ 393us) service_pool.cc:224] Handling call
0505 07:54:17.183188 (+7257302us) ts_tablet_manager.cc:1151] Deleting tablet 23fa401bb4634022bca3c6b23421e4d5
0505 07:54:17.183197 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.187044 (+ 3847us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.189935 (+ 2891us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.190979 (+ 1044us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7253298}
I20260505 07:54:17.191301 2268 tablet_replica.cc:333] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.191463 2268 raft_consensus.cc:2243] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.191793 2268 raft_consensus.cc:2272] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.192279 2268 ts_tablet_manager.cc:1916] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.206435 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a82ed2b87c487aac84194f2dcb0d6b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.206629 921 raft_consensus.cc:2468] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:17.208386 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 5191f9a357de4607aab6c7a8b4bcccdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.208338 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 66f069982138460da3b7ae7a0b4e3833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.208518 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet abaa596871bb4a82aefc317932e91d35 (table default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d [id=1fcae33a3a2d48e7b2000cc7b906c7f5]) successfully deleted
W20260505 07:54:17.208807 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e8bf49b787ef4de095448751464708a4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e8bf49b787ef4de095448751464708a4 already in progress: creating tablet
I20260505 07:54:17.209165 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 4ba2cc218300472cb5fd6dc8cfb7811c (table default.Table_27c52b9b060d4dd5a80799a987d5b7c9 [id=c41c8ca8836b4b4d96bc2f1129279f06]) successfully deleted
I20260505 07:54:17.209568 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 212e496d564a4b5ebabb22504be76bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:17.211859 3144 raft_consensus.cc:2749] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:17.212360 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 148585us
I20260505 07:54:17.212713 2268 ts_tablet_manager.cc:1929] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.212819 2268 log.cc:1199] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/17327cee69a24e0cb493ff55f948bf6c
W20260505 07:54:17.212836 856 leader_election.cc:343] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: f88a5c0d0e2c436c8d8b6afc1f3c2edf
I20260505 07:54:17.213057 856 leader_election.cc:304] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:17.213181 2268 ts_tablet_manager.cc:1950] T 17327cee69a24e0cb493ff55f948bf6c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.215579 1355 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_359f92a4ff554e3e80468261b0e86966" } new_table_name: "default.Table_61a6fc82edd5408f991f40a8d15394a5" modify_external_catalogs: true
I20260505 07:54:17.216429 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 201) took 7290 ms. Trace:
I20260505 07:54:17.216498 2268 rpcz_store.cc:276] 0505 07:54:09.926139 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.926450 (+ 311us) service_pool.cc:224] Handling call
0505 07:54:17.191267 (+7264817us) ts_tablet_manager.cc:1151] Deleting tablet 17327cee69a24e0cb493ff55f948bf6c
0505 07:54:17.191274 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.195155 (+ 3881us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.212671 (+ 17516us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.216413 (+ 3742us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7261335}
I20260505 07:54:17.216805 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 202) took 7290 ms. Trace:
I20260505 07:54:17.216861 2268 rpcz_store.cc:276] 0505 07:54:09.926904 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.927027 (+ 123us) service_pool.cc:224] Handling call
0505 07:54:17.216722 (+7289695us) ts_tablet_manager.cc:1151] Deleting tablet bde59d52fd234343b1ca13a7d4aae6bf
0505 07:54:17.216731 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.216795 (+ 64us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7286753}
I20260505 07:54:17.216979 1351 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7b4336be586a4fc1a8dada3651641fd9" } modify_external_catalogs: true
I20260505 07:54:17.217062 2268 tablet_replica.cc:333] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.217162 1351 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_7b4336be586a4fc1a8dada3651641fd9" } modify_external_catalogs: true
I20260505 07:54:17.217217 2268 raft_consensus.cc:2243] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.217339 2268 raft_consensus.cc:2272] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.217792 2268 ts_tablet_manager.cc:1916] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.236503 2268 ts_tablet_manager.cc:1929] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.236598 2268 log.cc:1199] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/12501fdf2cc5491cb1a06db490e70810
I20260505 07:54:17.236907 2268 ts_tablet_manager.cc:1950] T 12501fdf2cc5491cb1a06db490e70810 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.237460 1348 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 215) took 1745 ms. Trace:
I20260505 07:54:17.237562 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 192) took 7319 ms. Trace:
I20260505 07:54:17.237646 2268 rpcz_store.cc:276] 0505 07:54:09.918436 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.919212 (+ 776us) service_pool.cc:224] Handling call
0505 07:54:17.217024 (+7297812us) ts_tablet_manager.cc:1151] Deleting tablet 12501fdf2cc5491cb1a06db490e70810
0505 07:54:17.217032 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.229137 (+ 12105us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.236460 (+ 7323us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.237542 (+ 1082us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7286930}
I20260505 07:54:17.237927 2268 tablet_replica.cc:333] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.237610 1348 rpcz_store.cc:276] 0505 07:54:15.492391 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.068792 (+1576401us) service_pool.cc:224] Handling call
0505 07:54:17.069570 (+ 778us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:17.069963 (+ 393us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:17.237215 (+167252us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:17.237318 (+ 103us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:17.237409 (+ 91us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.073078 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.073151 (+ 73us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.073155 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:17.073158 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:17.073214 (+ 56us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.073222 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.073226 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.073273 (+ 47us) tablet.cc:701] Row locks acquired
0505 07:54:17.073275 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.073294 (+ 19us) write_op.cc:270] Start()
0505 07:54:17.073326 (+ 32us) write_op.cc:276] Timestamp: P: 1777967657073260 usec, L: 0
0505 07:54:17.073328 (+ 2us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.073452 (+ 124us) log.cc:844] Serialized 433 byte log entry
0505 07:54:17.213641 (+140189us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.236561 (+ 22920us) write_op.cc:301] APPLY: starting
0505 07:54:17.236577 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.236621 (+ 44us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.236649 (+ 28us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.236787 (+ 138us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.236829 (+ 42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:17.236841 (+ 12us) write_op.cc:312] APPLY: finished
0505 07:54:17.236935 (+ 94us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.237008 (+ 73us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.237043 (+ 35us) write_op.cc:454] Released schema lock
0505 07:54:17.237056 (+ 13us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.237136 (+ 80us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.237140 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":22860,"apply.run_cpu_time_us":566,"apply.run_wall_time_us":565,"num_ops":2,"prepare.queue_time_us":2756,"prepare.run_cpu_time_us":412,"prepare.run_wall_time_us":411,"replication_time_us":140278}]]}
I20260505 07:54:17.238121 2268 raft_consensus.cc:2243] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.238243 1348 master_service.cc:954] ReplaceTablet: received request to replace tablet 83e62b7af9b847b7a9005e4345905c0d from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:17.238294 2268 raft_consensus.cc:2272] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.238651 1353 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 152176572c904b8296073afad0585817
I20260505 07:54:17.238775 2268 ts_tablet_manager.cc:1916] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.239214 1353 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 205) took 2486 ms. Trace:
I20260505 07:54:17.239302 1353 rpcz_store.cc:276] 0505 07:54:14.752690 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.986313 (+2233623us) service_pool.cc:224] Handling call
0505 07:54:16.986776 (+ 463us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:16.986918 (+ 142us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:16.986988 (+ 70us) catalog_manager.cc:2824] Locking tablets
0505 07:54:17.116091 (+129103us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:17.238475 (+122384us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:17.238558 (+ 83us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:17.238603 (+ 45us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:17.239164 (+ 561us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.116811 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.116919 (+ 108us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.116924 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:17.116926 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.117038 (+ 112us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.117049 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.117053 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.117148 (+ 95us) tablet.cc:701] Row locks acquired
0505 07:54:17.117151 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:17.117183 (+ 32us) write_op.cc:270] Start()
0505 07:54:17.117216 (+ 33us) write_op.cc:276] Timestamp: P: 1777967657117141 usec, L: 0
0505 07:54:17.117220 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.214137 (+ 96917us) log.cc:844] Serialized 725 byte log entry
0505 07:54:17.214356 (+ 219us) spinlock_profiling.cc:243] Waited 96.7 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65bae8f22 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac
0505 07:54:17.236464 (+ 22108us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.237214 (+ 750us) write_op.cc:301] APPLY: starting
0505 07:54:17.237227 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.237263 (+ 36us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.237265 (+ 2us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.237331 (+ 66us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.237357 (+ 26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:17.237363 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:17.237434 (+ 71us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.237489 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.237503 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:17.237510 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":699,"apply.run_cpu_time_us":359,"apply.run_wall_time_us":358,"num_ops":2,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":1796,"prepare.run_wall_time_us":97607,"raft.queue_time_us":3910,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":242,"replication_time_us":119212,"spinlock_wait_cycles":222369664}]]}
I20260505 07:54:17.240739 1350 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 209) took 1967 ms. Trace:
I20260505 07:54:17.240851 1350 rpcz_store.cc:276] 0505 07:54:15.273526 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.018336 (+1744810us) service_pool.cc:224] Handling call
0505 07:54:17.050519 (+ 32183us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:17.083899 (+ 33380us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:17.240509 (+156610us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:17.240606 (+ 97us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:17.240687 (+ 81us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.214456 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.214609 (+ 153us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.214615 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:17.214617 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.214717 (+ 100us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.214732 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.214739 (+ 7us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.214839 (+ 100us) tablet.cc:701] Row locks acquired
0505 07:54:17.214844 (+ 5us) write_op.cc:260] PREPARE: finished
0505 07:54:17.214891 (+ 47us) write_op.cc:270] Start()
0505 07:54:17.214927 (+ 36us) write_op.cc:276] Timestamp: P: 1777967657214827 usec, L: 0
0505 07:54:17.214932 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.215099 (+ 167us) log.cc:844] Serialized 433 byte log entry
0505 07:54:17.237912 (+ 22813us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.238051 (+ 139us) write_op.cc:301] APPLY: starting
0505 07:54:17.238070 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.238106 (+ 36us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.238142 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.238237 (+ 95us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.238266 (+ 29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:17.238272 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:17.238329 (+ 57us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.238366 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.238382 (+ 16us) write_op.cc:454] Released schema lock
0505 07:54:17.238391 (+ 9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":82,"apply.run_cpu_time_us":406,"apply.run_wall_time_us":406,"num_ops":2,"prepare.queue_time_us":95956,"raft.queue_time_us":3417,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"replication_time_us":22852,"threads_started":1}]]}
I20260505 07:54:17.242398 3149 raft_consensus.cc:2749] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:17.245428 2268 ts_tablet_manager.cc:1929] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.247359 744 tablet_service.cc:1558] Processing DeleteTablet for tablet 152176572c904b8296073afad0585817 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.258080 2856 consensus_queue.cc:1048] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.258988 2276 raft_consensus.cc:2243] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.259295 2276 raft_consensus.cc:2272] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.259606 2268 log.cc:1199] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/a7d8265c05b74b28959d0de41fad1fde
I20260505 07:54:17.259933 2276 ts_tablet_manager.cc:1916] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.260032 2268 ts_tablet_manager.cc:1950] T a7d8265c05b74b28959d0de41fad1fde P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.260769 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 203) took 7333 ms. Trace:
I20260505 07:54:17.260848 2268 rpcz_store.cc:276] 0505 07:54:09.927784 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.943711 (+ 15927us) service_pool.cc:224] Handling call
0505 07:54:17.237884 (+7294173us) ts_tablet_manager.cc:1151] Deleting tablet a7d8265c05b74b28959d0de41fad1fde
0505 07:54:17.237895 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.241760 (+ 3865us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.245373 (+ 3613us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.260734 (+ 15361us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7294002}
I20260505 07:54:17.261197 2268 tablet_replica.cc:333] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.261219 1350 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_db06b2a6ada74addb6f217ab26d73870" } new_table_name: "default.Table_90c98f62112f465294c60c56f6835a93" modify_external_catalogs: true
I20260505 07:54:17.261412 2268 raft_consensus.cc:2243] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.261575 2268 raft_consensus.cc:2272] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.262163 2268 ts_tablet_manager.cc:1916] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.264035 1353 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_9c422539fda7423c820e2450a3f7c8db" } new_table_name: "default.Table_a3cc6e739a71414f9f925aa759a07bd3" modify_external_catalogs: true
I20260505 07:54:17.266415 2276 ts_tablet_manager.cc:1929] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.266527 2276 log.cc:1199] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/c1a61c1d03d544c8aa01537ead97f83c
I20260505 07:54:17.266954 2276 ts_tablet_manager.cc:1950] T c1a61c1d03d544c8aa01537ead97f83c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.267297 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 102) took 9488 ms. Trace:
I20260505 07:54:17.267374 2276 rpcz_store.cc:276] 0505 07:54:07.778503 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.778551 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:17.142776 (+9364225us) ts_tablet_manager.cc:1151] Deleting tablet c1a61c1d03d544c8aa01537ead97f83c
0505 07:54:17.142785 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.263088 (+120303us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.266366 (+ 3278us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.267274 (+ 908us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115856,"tablet-delete.queue_time_us":9339767}
I20260505 07:54:17.268075 2276 tablet_replica.cc:333] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.268288 2276 raft_consensus.cc:2243] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.268424 2276 raft_consensus.cc:2272] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.268904 2276 ts_tablet_manager.cc:1916] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.273337 2268 ts_tablet_manager.cc:1929] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.273434 2268 log.cc:1199] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/067c32ecccb44bca8bf9cc3f6c229326
I20260505 07:54:17.273759 2268 ts_tablet_manager.cc:1950] T 067c32ecccb44bca8bf9cc3f6c229326 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.274443 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 204) took 7330 ms. Trace:
I20260505 07:54:17.274524 1352 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 8cf3c334ebb24956b50dbefc23d4d60b
I20260505 07:54:17.274541 2268 rpcz_store.cc:276] 0505 07:54:09.944015 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.944056 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:17.261144 (+7317088us) ts_tablet_manager.cc:1151] Deleting tablet 067c32ecccb44bca8bf9cc3f6c229326
0505 07:54:17.261156 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.265536 (+ 4380us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.273291 (+ 7755us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.274417 (+ 1126us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7316962}
I20260505 07:54:17.274847 2268 tablet_replica.cc:333] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.275099 1352 catalog_manager.cc:6741] ReplaceTablet: tablet 8cf3c334ebb24956b50dbefc23d4d60b deleted and replaced by tablet 260a8d41de464e429491ada4bfe2c761
I20260505 07:54:17.275228 1352 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 217) took 1221 ms. Trace:
I20260505 07:54:17.275332 1352 rpcz_store.cc:276] 0505 07:54:16.053804 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.080542 (+1026738us) service_pool.cc:224] Handling call
0505 07:54:17.275196 (+194654us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.260053 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.260217 (+ 164us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.260222 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:17.260224 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.260382 (+ 158us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.260397 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.260402 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.260488 (+ 86us) tablet.cc:701] Row locks acquired
0505 07:54:17.260490 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.260533 (+ 43us) write_op.cc:270] Start()
0505 07:54:17.260569 (+ 36us) write_op.cc:276] Timestamp: P: 1777967657260474 usec, L: 0
0505 07:54:17.260573 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.267009 (+ 6436us) log.cc:844] Serialized 622 byte log entry
0505 07:54:17.267231 (+ 222us) spinlock_profiling.cc:243] Waited 6.21 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 000000000099a74a 00007fd65bae8f22 00007fd65ced1c55 00007fd65ced0779 00007fd65ced8ba0 00007fd65ced89ad 0000000000aa4cac 00007fd64fc15f6f 00007fd64fc1928b 00007fd64fc19108 0000000000aa4cac
0505 07:54:17.270922 (+ 3691us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.273673 (+ 2751us) write_op.cc:301] APPLY: starting
0505 07:54:17.273692 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.273766 (+ 74us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.273771 (+ 5us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.273925 (+ 154us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.274027 (+ 102us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:17.274040 (+ 13us) write_op.cc:312] APPLY: finished
0505 07:54:17.274120 (+ 80us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.274178 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.274192 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:17.274198 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2620,"num_ops":2,"prepare.queue_time_us":140161,"prepare.run_cpu_time_us":1206,"prepare.run_wall_time_us":8280,"raft.queue_time_us":1006,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":6,"replication_time_us":10321,"spinlock_wait_cycles":14280064}]]}
I20260505 07:54:17.277315 2268 raft_consensus.cc:2243] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.277598 2268 raft_consensus.cc:2272] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.278110 2268 ts_tablet_manager.cc:1916] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.279661 1176 tablet_bootstrap.cc:492] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:17.279805 1176 ts_tablet_manager.cc:1403] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.134s user 0.013s sys 0.011s
I20260505 07:54:17.280545 1176 raft_consensus.cc:359] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.280723 1176 raft_consensus.cc:385] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.280774 1176 raft_consensus.cc:740] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.280982 1176 consensus_queue.cc:260] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.281512 1176 ts_tablet_manager.cc:1434] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:17.281831 1176 tablet_bootstrap.cc:492] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
W20260505 07:54:17.283175 1354 catalog_manager.cc:5863] Failed to send AlterTable RPC for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 06d87dc359cb48c2b5a6f5008f33427e consensus state has no leader
I20260505 07:54:17.283229 1176 tablet_bootstrap.cc:654] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.283820 2268 ts_tablet_manager.cc:1929] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.283918 2268 log.cc:1199] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/0db162dfad2448e8ac428b3d3fbe17f7
I20260505 07:54:17.284245 2268 ts_tablet_manager.cc:1950] T 0db162dfad2448e8ac428b3d3fbe17f7 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.284945 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 205) took 7340 ms. Trace:
I20260505 07:54:17.285027 2268 rpcz_store.cc:276] 0505 07:54:09.944299 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.944338 (+ 39us) service_pool.cc:224] Handling call
0505 07:54:17.274803 (+7330465us) ts_tablet_manager.cc:1151] Deleting tablet 0db162dfad2448e8ac428b3d3fbe17f7
0505 07:54:17.274811 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.281107 (+ 6296us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.283778 (+ 2671us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.284928 (+ 1150us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7330324}
I20260505 07:54:17.285346 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 206) took 7340 ms. Trace:
I20260505 07:54:17.285423 2268 rpcz_store.cc:276] 0505 07:54:09.944610 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.944650 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:17.285260 (+7340610us) ts_tablet_manager.cc:1151] Deleting tablet 8b57c1c618484d5d86a846fb982a2240
0505 07:54:17.285269 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.285333 (+ 64us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7340478}
I20260505 07:54:17.285663 2268 tablet_replica.cc:333] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.285861 2268 raft_consensus.cc:2243] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.285993 2268 raft_consensus.cc:2272] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.286453 2268 ts_tablet_manager.cc:1916] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:17.297268 990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260505 07:54:17.298086 744 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cf3c334ebb24956b50dbefc23d4d60b with delete_type TABLET_DATA_DELETED (replaced by tablet 260a8d41de464e429491ada4bfe2c761) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:17.298523 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 85738us
I20260505 07:54:17.303632 2447 ts_tablet_manager.cc:1929] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.303825 2447 log.cc:1199] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/f1058fcb9fa6457a85489049ca1d9ada
I20260505 07:54:17.304311 2447 ts_tablet_manager.cc:1950] T f1058fcb9fa6457a85489049ca1d9ada P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:17.305253 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 177) took 7752 ms. Trace:
I20260505 07:54:17.308301 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 27bacc97d7534284970f218cf68ac75b (table default.Table_db40b13ba59549c6bae350382395bceb [id=db770342dce84ba1ab28d029ddb65d78]) successfully deleted
W20260505 07:54:17.308562 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8eb7eaff8837485a8ae3ccb44c60471d because the tablet was not found. No further retry: Not found: Tablet not found: 8eb7eaff8837485a8ae3ccb44c60471d
I20260505 07:54:17.309630 2268 ts_tablet_manager.cc:1929] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260505 07:54:17.309867 1354 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.2:57579 (request call id 160) took 2008 ms (client timeout 1000 ms). Trace:
I20260505 07:54:17.309947 2268 log.cc:1199] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/62d4bc3154704ba2bfde83bca10f812d
I20260505 07:54:17.310312 2268 ts_tablet_manager.cc:1950] T 62d4bc3154704ba2bfde83bca10f812d P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.310956 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 207) took 7366 ms. Trace:
I20260505 07:54:17.311044 2268 rpcz_store.cc:276] 0505 07:54:09.944911 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.944949 (+ 38us) service_pool.cc:224] Handling call
0505 07:54:17.285624 (+7340675us) ts_tablet_manager.cc:1151] Deleting tablet 62d4bc3154704ba2bfde83bca10f812d
0505 07:54:17.285633 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.305787 (+ 20154us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.309559 (+ 3772us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.310939 (+ 1380us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7340550}
I20260505 07:54:17.311317 2268 tablet_replica.cc:333] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.311503 2268 raft_consensus.cc:2243] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.311662 2268 raft_consensus.cc:2272] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.303562 2276 ts_tablet_manager.cc:1929] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:17.312157 2268 ts_tablet_manager.cc:1916] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.312196 2276 log.cc:1199] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/d77ff90ec2ff4b6eb1795d149459c23a
I20260505 07:54:17.312460 2276 ts_tablet_manager.cc:1950] T d77ff90ec2ff4b6eb1795d149459c23a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.312738 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 103) took 9534 ms. Trace:
I20260505 07:54:17.312811 2276 rpcz_store.cc:276] 0505 07:54:07.778754 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.778797 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:17.268025 (+9489228us) ts_tablet_manager.cc:1151] Deleting tablet d77ff90ec2ff4b6eb1795d149459c23a
0505 07:54:17.268036 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.298249 (+ 30213us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.303515 (+ 5266us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.312721 (+ 9206us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9464978}
I20260505 07:54:17.313077 2276 tablet_replica.cc:333] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.313252 2276 raft_consensus.cc:2243] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.313479 2276 raft_consensus.cc:2272] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.313993 2276 ts_tablet_manager.cc:1916] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:17.316555 1354 rpcz_store.cc:269] 0505 07:54:15.301342 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.719402 (+418060us) service_pool.cc:224] Handling call
0505 07:54:16.344710 (+625308us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.344716 (+ 6us) write_op.cc:454] Released schema lock
0505 07:54:17.309475 (+964759us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.337241 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.337351 (+ 110us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.337357 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:16.337359 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:16.337515 (+ 156us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.337531 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.337538 (+ 7us) tablet.cc:685] Acquiring locks for 4 operations
0505 07:54:16.337669 (+ 131us) tablet.cc:701] Row locks acquired
0505 07:54:16.337672 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:16.337706 (+ 34us) write_op.cc:270] Start()
0505 07:54:16.337735 (+ 29us) write_op.cc:276] Timestamp: P: 1777967656337611 usec, L: 0
0505 07:54:16.337738 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.337909 (+ 171us) log.cc:844] Serialized 1650 byte log entry
0505 07:54:16.341376 (+ 3467us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.341504 (+ 128us) write_op.cc:301] APPLY: starting
0505 07:54:16.341520 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.341591 (+ 71us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.341626 (+ 35us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.341744 (+ 118us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.341787 (+ 43us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0505 07:54:16.341798 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:16.341897 (+ 99us) log.cc:844] Serialized 41 byte log entry
0505 07:54:16.341962 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.341988 (+ 26us) write_op.cc:454] Released schema lock
0505 07:54:16.341995 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":257,"child_traces":[["op",{"apply.queue_time_us":79,"apply.run_cpu_time_us":565,"apply.run_wall_time_us":565,"mutex_wait_us":24693,"num_ops":4,"prepare.queue_time_us":24917,"prepare.run_cpu_time_us":770,"prepare.run_wall_time_us":770,"raft.queue_time_us":319,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":198,"replication_time_us":3608}]]}
I20260505 07:54:17.317067 2268 ts_tablet_manager.cc:1929] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.317167 2268 log.cc:1199] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/141665fcd59048bf926acd0287f1853e
I20260505 07:54:17.317462 2268 ts_tablet_manager.cc:1950] T 141665fcd59048bf926acd0287f1853e P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.318050 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 208) took 7372 ms. Trace:
I20260505 07:54:17.318113 2268 rpcz_store.cc:276] 0505 07:54:09.945187 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.945225 (+ 38us) service_pool.cc:224] Handling call
0505 07:54:17.311277 (+7366052us) ts_tablet_manager.cc:1151] Deleting tablet 141665fcd59048bf926acd0287f1853e
0505 07:54:17.311287 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.314539 (+ 3252us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.317031 (+ 2492us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.318035 (+ 1004us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7365930}
I20260505 07:54:17.318341 2268 tablet_replica.cc:333] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.318497 2268 raft_consensus.cc:2243] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.318723 2268 raft_consensus.cc:2272] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.319211 2268 ts_tablet_manager.cc:1916] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.327571 2276 ts_tablet_manager.cc:1929] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.332969 2276 log.cc:1199] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/bc9ae6404dd64bd8bf4898768e94e4f1
I20260505 07:54:17.333349 2276 ts_tablet_manager.cc:1950] T bc9ae6404dd64bd8bf4898768e94e4f1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.333542 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 78a6470720984761a8606abeafbef992 (table default.Table_5cd903011cdc44a392680507ed9088c8 [id=a3c321e46ecd4b07809a9c663305afb7]) successfully deleted
I20260505 07:54:17.333664 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 104) took 9554 ms. Trace:
I20260505 07:54:17.333742 2276 rpcz_store.cc:276] 0505 07:54:07.779007 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.779053 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:17.313036 (+9533983us) ts_tablet_manager.cc:1151] Deleting tablet bc9ae6404dd64bd8bf4898768e94e4f1
0505 07:54:17.313045 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.324230 (+ 11185us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.327521 (+ 3291us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.333648 (+ 6127us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9509941}
I20260505 07:54:17.334039 2276 tablet_replica.cc:333] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.334226 2276 raft_consensus.cc:2243] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.334429 2276 raft_consensus.cc:2272] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.334766 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f5b72f58a24db0b58c15b47227bc60" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:17.334980 2276 ts_tablet_manager.cc:1916] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.335544 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 202a237ca580482c98803683a77d96d2 (table default.Table_2f6f14275b8847be96f68a761156a854 [id=9c830b1f855348299c40b40610589233]) successfully deleted
I20260505 07:54:17.332820 1172 ts_tablet_manager.cc:1434] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.363s user 0.005s sys 0.025s
I20260505 07:54:17.335904 1172 tablet_bootstrap.cc:492] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.337517 1172 tablet_bootstrap.cc:654] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.339782 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 62d4bc3154704ba2bfde83bca10f812d (table default.Table_82a545edb422426ab7eaeb4ae68a1a3f [id=5a1cf8871df74960bb29f3288ea4826c]) successfully deleted
I20260505 07:54:17.340029 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 8b57c1c618484d5d86a846fb982a2240 (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]) successfully deleted
I20260505 07:54:17.340768 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 003cd4472b424aacbba8c5f89900e228 (table default.Table_bd283510c1f14a9fb29a9870d33c9971 [id=ae40934b942a4e0e8ac882d9dab6686f]) successfully deleted
I20260505 07:54:17.340960 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet b0b9e8eb767a4eceb583ff9b47000d58 (table default.Table_49c6ff5135b14b6b8437bb753e1b6342 [id=169b57b0e527456cb435bd97c47782bb]) successfully deleted
I20260505 07:54:17.342350 1181 tablet_bootstrap.cc:492] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:17.342475 1181 ts_tablet_manager.cc:1403] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.369s user 0.042s sys 0.074s
I20260505 07:54:17.343230 1181 raft_consensus.cc:359] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.343407 1181 raft_consensus.cc:385] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.343474 1181 raft_consensus.cc:740] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.343755 1181 consensus_queue.cc:260] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.344190 1181 ts_tablet_manager.cc:1434] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:17.344434 1181 tablet_bootstrap.cc:492] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:17.346118 1181 tablet_bootstrap.cc:654] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.346472 2268 ts_tablet_manager.cc:1929] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.346567 2268 log.cc:1199] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/feeafd614b3e4281a8e45b950e699b63
I20260505 07:54:17.346925 2268 ts_tablet_manager.cc:1950] T feeafd614b3e4281a8e45b950e699b63 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.347589 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 209) took 7402 ms. Trace:
I20260505 07:54:17.347697 2268 rpcz_store.cc:276] 0505 07:54:09.945510 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.945551 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:17.318301 (+7372750us) ts_tablet_manager.cc:1151] Deleting tablet feeafd614b3e4281a8e45b950e699b63
0505 07:54:17.318310 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.333988 (+ 15678us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.346424 (+ 12436us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.347573 (+ 1149us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7372604}
I20260505 07:54:17.347994 2268 tablet_replica.cc:333] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.348191 2268 raft_consensus.cc:2243] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.348351 2268 raft_consensus.cc:2272] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.348838 2268 ts_tablet_manager.cc:1916] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.354241 2268 ts_tablet_manager.cc:1929] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.354347 2268 log.cc:1199] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/b5a0cbf9c3d74cea92920189da332c1c
I20260505 07:54:17.354686 2268 ts_tablet_manager.cc:1950] T b5a0cbf9c3d74cea92920189da332c1c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.355317 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 210) took 7409 ms. Trace:
I20260505 07:54:17.355396 2268 rpcz_store.cc:276] 0505 07:54:09.945872 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.945913 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:17.347955 (+7402042us) ts_tablet_manager.cc:1151] Deleting tablet b5a0cbf9c3d74cea92920189da332c1c
0505 07:54:17.347965 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.351658 (+ 3693us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.354208 (+ 2550us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.355301 (+ 1093us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7401908}
I20260505 07:54:17.356635 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet d622612bdc404727a27b83769bf39738 (table default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]) successfully deleted
I20260505 07:54:17.357275 1181 tablet_bootstrap.cc:492] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:17.357345 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet c1a61c1d03d544c8aa01537ead97f83c (table default.Table_575ab5b128ee4a1baebbe1f90baf005a [id=cdb574c7a4e14a5ea7f3c8b036a8c336]) successfully deleted
I20260505 07:54:17.357399 1181 ts_tablet_manager.cc:1403] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.005s
I20260505 07:54:17.358134 1181 raft_consensus.cc:359] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.358314 1181 raft_consensus.cc:385] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.358381 1181 raft_consensus.cc:740] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.358569 1181 consensus_queue.cc:260] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.359731 2276 ts_tablet_manager.cc:1929] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.359833 2276 log.cc:1199] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/3f1feb0387bc4afead72346398dbe9e5
I20260505 07:54:17.360179 2276 ts_tablet_manager.cc:1950] T 3f1feb0387bc4afead72346398dbe9e5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.360436 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 105) took 9581 ms. Trace:
I20260505 07:54:17.360498 2276 rpcz_store.cc:276] 0505 07:54:07.779265 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.780544 (+ 1279us) service_pool.cc:224] Handling call
0505 07:54:17.333997 (+9553453us) ts_tablet_manager.cc:1151] Deleting tablet 3f1feb0387bc4afead72346398dbe9e5
0505 07:54:17.334006 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.349088 (+ 15082us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.359682 (+ 10594us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.360422 (+ 740us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9530855}
I20260505 07:54:17.360759 2276 tablet_replica.cc:333] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.360930 2276 raft_consensus.cc:2243] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.361069 2276 raft_consensus.cc:2272] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.361567 2276 ts_tablet_manager.cc:1916] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.359138 1181 ts_tablet_manager.cc:1434] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:17.364039 2268 tablet_replica.cc:333] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.364214 2268 raft_consensus.cc:2243] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.364348 2268 raft_consensus.cc:2272] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.364459 1172 tablet_bootstrap.cc:492] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.364627 1172 ts_tablet_manager.cc:1403] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.003s
I20260505 07:54:17.364822 2268 ts_tablet_manager.cc:1916] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.365335 1172 raft_consensus.cc:359] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.365509 1172 raft_consensus.cc:385] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.365577 1172 raft_consensus.cc:740] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.365790 1172 consensus_queue.cc:260] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.366271 1172 ts_tablet_manager.cc:1434] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:17.366570 1172 tablet_bootstrap.cc:492] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.367264 1181 tablet_bootstrap.cc:492] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:17.369213 1181 tablet_bootstrap.cc:654] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.371227 2268 ts_tablet_manager.cc:1929] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.371316 2268 log.cc:1199] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/f92c53c68a4b4e319e98b84128e8e6a4
I20260505 07:54:17.371640 2268 ts_tablet_manager.cc:1950] T f92c53c68a4b4e319e98b84128e8e6a4 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.372225 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 211) took 7426 ms. Trace:
I20260505 07:54:17.372308 2268 rpcz_store.cc:276] 0505 07:54:09.946198 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.946238 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:17.364002 (+7417764us) ts_tablet_manager.cc:1151] Deleting tablet f92c53c68a4b4e319e98b84128e8e6a4
0505 07:54:17.364010 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.368216 (+ 4206us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.371190 (+ 2974us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.372211 (+ 1021us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7417621}
I20260505 07:54:17.372555 2268 tablet_replica.cc:333] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.372705 2268 raft_consensus.cc:2243] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.372843 2268 raft_consensus.cc:2272] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.373255 2268 ts_tablet_manager.cc:1916] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.374503 2276 ts_tablet_manager.cc:1929] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.374622 2276 log.cc:1199] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/d3d11bd19f074bbab101448b45426b7a
I20260505 07:54:17.375056 2276 ts_tablet_manager.cc:1950] T d3d11bd19f074bbab101448b45426b7a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.375367 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 106) took 9594 ms. Trace:
I20260505 07:54:17.375471 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet d77ff90ec2ff4b6eb1795d149459c23a (table default.Table_1ff5286907c6464c8addf47c70eef7ef [id=f608e93cbc64485297c6dc2609a6eb87]) successfully deleted
I20260505 07:54:17.378893 1172 tablet_bootstrap.cc:654] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.381909 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.382184 921 raft_consensus.cc:2468] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:17.375443 2276 rpcz_store.cc:276] 0505 07:54:07.780779 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:07.781146 (+ 367us) service_pool.cc:224] Handling call
0505 07:54:17.360720 (+9579574us) ts_tablet_manager.cc:1151] Deleting tablet d3d11bd19f074bbab101448b45426b7a
0505 07:54:17.360729 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.371138 (+ 10409us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.374449 (+ 3311us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.375350 (+ 901us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9557540}
I20260505 07:54:17.386878 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7887c3462f74740b314e3bec36e280d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.387233 921 raft_consensus.cc:2468] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:17.387069 2276 tablet_replica.cc:333] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.387964 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 212e496d564a4b5ebabb22504be76bf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.388418 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 66f069982138460da3b7ae7a0b4e3833 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.388033 2276 raft_consensus.cc:2243] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.388901 2276 raft_consensus.cc:2272] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.389500 2276 ts_tablet_manager.cc:1916] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.393254 1172 tablet_bootstrap.cc:492] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.393405 1172 ts_tablet_manager.cc:1403] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.002s
I20260505 07:54:17.393540 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 3f1feb0387bc4afead72346398dbe9e5 (table default.Table_dcf56067c2654eff8773e49d0ddf5427 [id=a9ef91bfcff640abb23140249a59fcf4]) successfully deleted
I20260505 07:54:17.394167 1172 raft_consensus.cc:359] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.394344 1172 raft_consensus.cc:385] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.394410 1172 raft_consensus.cc:740] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.394665 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 5191f9a357de4607aab6c7a8b4bcccdb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.394660 1172 consensus_queue.cc:260] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.395102 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 152176572c904b8296073afad0585817 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.395314 1172 ts_tablet_manager.cc:1434] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:17.395634 1172 tablet_bootstrap.cc:492] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.395617 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 03f31efbc98c484c950c70d5e84d2136 (table default.Table_6ff000247d324fe8b55b1398e48cb49a [id=8903351259c347a1925769663d34d173]) successfully deleted
I20260505 07:54:17.395818 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 323269e708964848a94c64e6a781a860 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]) successfully deleted
I20260505 07:54:17.397329 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 0459f2d45a2d44f1a73e0a0fff53866c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.397389 1172 tablet_bootstrap.cc:654] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.398921 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cf3c334ebb24956b50dbefc23d4d60b with delete_type TABLET_DATA_DELETED (replaced by tablet 260a8d41de464e429491ada4bfe2c761) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.399324 1181 tablet_bootstrap.cc:492] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:17.399462 1181 ts_tablet_manager.cc:1403] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.032s user 0.004s sys 0.004s
I20260505 07:54:17.400100 2268 ts_tablet_manager.cc:1929] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.400202 2268 log.cc:1199] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/bc9ae6404dd64bd8bf4898768e94e4f1
I20260505 07:54:17.400369 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 93d878ae0ddb42319560f74e0a490321 with delete_type TABLET_DATA_DELETED (replaced by tablet bc9ae6404dd64bd8bf4898768e94e4f1) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.400388 1181 raft_consensus.cc:359] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.400558 1181 raft_consensus.cc:385] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.400575 2268 ts_tablet_manager.cc:1950] T bc9ae6404dd64bd8bf4898768e94e4f1 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.400709 1181 raft_consensus.cc:740] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.400987 1181 consensus_queue.cc:260] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.401291 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 212) took 7454 ms. Trace:
I20260505 07:54:17.401388 2268 rpcz_store.cc:276] 0505 07:54:09.946550 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.946591 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:17.372522 (+7425931us) ts_tablet_manager.cc:1151] Deleting tablet bc9ae6404dd64bd8bf4898768e94e4f1
0505 07:54:17.372530 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.385455 (+ 12925us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.400048 (+ 14593us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.401274 (+ 1226us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7425806}
I20260505 07:54:17.401625 1181 ts_tablet_manager.cc:1434] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:17.401708 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 213) took 7454 ms. Trace:
I20260505 07:54:17.401799 2268 rpcz_store.cc:276] 0505 07:54:09.946903 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.946944 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:17.401622 (+7454678us) ts_tablet_manager.cc:1151] Deleting tablet 0baf83b155494fddb3c25144608706f7
0505 07:54:17.401632 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.401693 (+ 61us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7454536}
I20260505 07:54:17.401923 1181 tablet_bootstrap.cc:492] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:17.402038 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 323269e708964848a94c64e6a781a860 with delete_type TABLET_DATA_DELETED (replaced by tablet e010f23c4df647ab863e03c3cfed7898) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.402066 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 214) took 7454 ms. Trace:
I20260505 07:54:17.402138 2268 rpcz_store.cc:276] 0505 07:54:09.947241 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.947281 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:17.401997 (+7454716us) ts_tablet_manager.cc:1151] Deleting tablet d8c9eaaef250461ca7a37d0cf42b07ee
0505 07:54:17.402006 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.402053 (+ 47us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7454586}
I20260505 07:54:17.402354 2268 tablet_replica.cc:333] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.402515 2268 raft_consensus.cc:2243] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.402654 2268 raft_consensus.cc:2272] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.403151 2268 ts_tablet_manager.cc:1916] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.403394 896 tablet_service.cc:1558] Processing DeleteTablet for tablet d8c9eaaef250461ca7a37d0cf42b07ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.403667 1181 tablet_bootstrap.cc:654] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.404327 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 9959be9f83c2444282d012b61eef49b5 (table default.Table_ae34ec4a8b52453dba7158ce065f4798 [id=8314d2bc7c1a401db1d6fac29fff5c3f]) successfully deleted
W20260505 07:54:17.404577 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 6da10815e0dc41a1a07b6786882fffd3 because the tablet was not found. No further retry: Not found: Tablet not found: 6da10815e0dc41a1a07b6786882fffd3
I20260505 07:54:17.404757 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 8a60c015136246a3b02f70254c0bb67f (table default.Table_d9857ceb9b4d48dab8120e696f98e0e5 [id=65a99c8ba7b14ed0aec12382a2c8095c]) successfully deleted
I20260505 07:54:17.404907 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 9044763bed1a41739ecce4def4cca1fe (table default.Table_1523b3ae7cd94c599e60a187bc677fc0 [id=bd575286f4e94df3b335b5e79d8f971c]) successfully deleted
I20260505 07:54:17.405081 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 06904a54b1c5463da31af0bc00fb606f (table default.Table_577ff393d2af447a915f4460ab7692f1 [id=96336409bde3400fb18cb06e0d84e99c]) successfully deleted
I20260505 07:54:17.405334 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 26f3544e74d941d286a2a0317b941ab6 (table default.Table_82c755fca0b64337a44a63876e04fa68 [id=ded6ca9533da4aa197bbd8c9b4a72066]) successfully deleted
I20260505 07:54:17.405550 894 tablet_service.cc:1558] Processing DeleteTablet for tablet e5eaa91ecc1f43d18b313b6be4cc211e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.406579 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 0baf83b155494fddb3c25144608706f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.407094 899 tablet_service.cc:1558] Processing DeleteTablet for tablet feeafd614b3e4281a8e45b950e699b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.407253 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 397d33e908af4a0bb8dde26d273be01a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.407378 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 60772a9697ba417b879dc6025eeb8eec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.407493 887 tablet_service.cc:1558] Processing DeleteTablet for tablet b5a0cbf9c3d74cea92920189da332c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.407693 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet bc9ae6404dd64bd8bf4898768e94e4f1 (table default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]) successfully deleted
I20260505 07:54:17.408900 2268 ts_tablet_manager.cc:1929] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.408984 2268 log.cc:1199] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/6fe19306f7df49fdb771d21f6560e697
I20260505 07:54:17.409309 2268 ts_tablet_manager.cc:1950] T 6fe19306f7df49fdb771d21f6560e697 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.409385 2276 ts_tablet_manager.cc:1929] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:17.409480 2276 log.cc:1199] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/25072055f5534078adf5b5025614f951
I20260505 07:54:17.409816 2276 ts_tablet_manager.cc:1950] T 25072055f5534078adf5b5025614f951 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.409989 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 216) took 7462 ms. Trace:
I20260505 07:54:17.410076 2268 rpcz_store.cc:276] 0505 07:54:09.947959 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.947997 (+ 38us) service_pool.cc:224] Handling call
0505 07:54:17.402316 (+7454319us) ts_tablet_manager.cc:1151] Deleting tablet 6fe19306f7df49fdb771d21f6560e697
0505 07:54:17.402323 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.406189 (+ 3866us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.408856 (+ 2667us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.409969 (+ 1113us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7454183}
I20260505 07:54:17.410385 2268 tablet_replica.cc:333] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.410588 2268 raft_consensus.cc:2243] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.410766 2268 raft_consensus.cc:2272] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.410861 1172 tablet_bootstrap.cc:492] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.411016 1172 ts_tablet_manager.cc:1403] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.000s
I20260505 07:54:17.411350 2268 ts_tablet_manager.cc:1916] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.309396 3004 consensus_queue.cc:1048] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.311393 1176 tablet_bootstrap.cc:492] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:17.411569 1176 ts_tablet_manager.cc:1403] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.130s user 0.002s sys 0.004s
I20260505 07:54:17.411836 1172 raft_consensus.cc:359] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.411998 1172 raft_consensus.cc:385] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.412057 1172 raft_consensus.cc:740] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.412248 1172 consensus_queue.cc:260] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.412365 1176 raft_consensus.cc:359] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.412526 1176 raft_consensus.cc:385] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.412583 1176 raft_consensus.cc:740] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.412837 1172 ts_tablet_manager.cc:1434] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:17.412829 1176 consensus_queue.cc:260] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.413239 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 133) took 9301 ms. Trace:
I20260505 07:54:17.413354 1176 ts_tablet_manager.cc:1434] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:17.413316 2276 rpcz_store.cc:276] 0505 07:54:08.112272 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:08.112326 (+ 54us) service_pool.cc:224] Handling call
0505 07:54:17.387029 (+9274703us) ts_tablet_manager.cc:1151] Deleting tablet 25072055f5534078adf5b5025614f951
0505 07:54:17.387038 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.406028 (+ 18990us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.409334 (+ 3306us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.413224 (+ 3890us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9274546}
I20260505 07:54:17.413599 2276 tablet_replica.cc:333] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.413657 1176 tablet_bootstrap.cc:492] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:17.413777 2276 raft_consensus.cc:2243] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.413913 2276 raft_consensus.cc:2272] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.414371 2276 ts_tablet_manager.cc:1916] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.415120 1176 tablet_bootstrap.cc:654] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:17.321005 717 consensus_peers.cc:597] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9044763bed1a41739ecce4def4cca1fe. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:17.415438 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 94472.1us
I20260505 07:54:17.415863 1172 tablet_bootstrap.cc:492] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.367391 3013 consensus_queue.cc:1048] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.417577 1172 tablet_bootstrap.cc:654] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.419250 1181 tablet_bootstrap.cc:492] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:17.419449 1181 ts_tablet_manager.cc:1403] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.003s
I20260505 07:54:17.420516 1181 raft_consensus.cc:359] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.420755 1181 raft_consensus.cc:385] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.420863 1181 raft_consensus.cc:740] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.421110 1181 consensus_queue.cc:260] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.421828 1181 ts_tablet_manager.cc:1434] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:17.422168 1181 tablet_bootstrap.cc:492] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:17.369249 2831 consensus_queue.cc:1048] T 429ccdb56f0b456483cf1debb5de0beb P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.424264 1181 tablet_bootstrap.cc:654] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.429579 2268 ts_tablet_manager.cc:1929] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:17.429731 2268 log.cc:1199] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/5683d7366de84653ace56f0edcd3818c
I20260505 07:54:17.430197 2268 ts_tablet_manager.cc:1950] T 5683d7366de84653ace56f0edcd3818c P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.430990 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 218) took 7077 ms. Trace:
I20260505 07:54:17.431151 2268 rpcz_store.cc:276] 0505 07:54:10.353606 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.355630 (+ 2024us) service_pool.cc:224] Handling call
0505 07:54:17.410334 (+7054704us) ts_tablet_manager.cc:1151] Deleting tablet 5683d7366de84653ace56f0edcd3818c
0505 07:54:17.410344 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.414462 (+ 4118us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.429532 (+ 15070us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.430973 (+ 1441us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7054543}
I20260505 07:54:17.431525 2268 tablet_replica.cc:333] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.431846 2268 raft_consensus.cc:2243] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.432060 2268 raft_consensus.cc:2272] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.432654 2268 ts_tablet_manager.cc:1916] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.432965 1349 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 9c4fc18f6faa488882d9bffb30793cdd
I20260505 07:54:17.433604 1349 catalog_manager.cc:6741] ReplaceTablet: tablet 9c4fc18f6faa488882d9bffb30793cdd deleted and replaced by tablet 57b5378ec4ae40448311d54c79ea33b0
I20260505 07:54:17.433888 1349 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 216) took 1904 ms. Trace:
W20260505 07:54:17.434644 1355 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 90612f0afb4949f491e625c61e40d247 consensus state has no leader
I20260505 07:54:17.434060 1349 rpcz_store.cc:276] 0505 07:54:15.529826 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.074871 (+1545045us) service_pool.cc:224] Handling call
0505 07:54:17.433851 (+358980us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.268365 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.268465 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.268471 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:17.268473 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.268583 (+ 110us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.268595 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.268599 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.268697 (+ 98us) tablet.cc:701] Row locks acquired
0505 07:54:17.268699 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.268734 (+ 35us) write_op.cc:270] Start()
0505 07:54:17.268765 (+ 31us) write_op.cc:276] Timestamp: P: 1777967657268676 usec, L: 0
0505 07:54:17.268769 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.268924 (+ 155us) log.cc:844] Serialized 622 byte log entry
0505 07:54:17.403965 (+135041us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.405652 (+ 1687us) write_op.cc:301] APPLY: starting
0505 07:54:17.405674 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.405719 (+ 45us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.405755 (+ 36us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.405862 (+ 107us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.405913 (+ 51us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:17.405919 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:17.406032 (+ 113us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.406111 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.406135 (+ 24us) write_op.cc:454] Released schema lock
0505 07:54:17.406145 (+ 10us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.406224 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.406228 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1621,"apply.run_cpu_time_us":571,"apply.run_wall_time_us":570,"num_ops":2,"prepare.queue_time_us":144196,"prepare.run_cpu_time_us":651,"prepare.run_wall_time_us":650,"replication_time_us":135164}]]}
I20260505 07:54:17.435335 1349 master_service.cc:954] ReplaceTablet: received request to replace tablet 99850198f4d04607b52953342daa753e from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:17.436605 2447 rpcz_store.cc:276] 0505 07:54:09.552662 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.552705 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:16.925350 (+7372645us) ts_tablet_manager.cc:1151] Deleting tablet f1058fcb9fa6457a85489049ca1d9ada
0505 07:54:17.140895 (+215545us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.196134 (+ 55239us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.303547 (+107413us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.305231 (+ 1684us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7366301}
I20260505 07:54:17.437142 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 178) took 7884 ms. Trace:
I20260505 07:54:17.437229 2447 rpcz_store.cc:276] 0505 07:54:09.552901 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.552966 (+ 65us) service_pool.cc:224] Handling call
0505 07:54:17.436975 (+7884009us) ts_tablet_manager.cc:1151] Deleting tablet b12d2d2b866348d781209fa787da7d7e
0505 07:54:17.436991 (+ 16us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.437122 (+ 131us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7877876}
I20260505 07:54:17.437525 2447 tablet_replica.cc:333] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:17.437773 2447 raft_consensus.cc:2243] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.437942 2447 raft_consensus.cc:2272] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:17.407277 718 consensus_peers.cc:597] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b5a0cbf9c3d74cea92920189da332c1c. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:17.440147 2276 ts_tablet_manager.cc:1929] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:17.440261 2276 log.cc:1199] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/72da22d790624b07bfca6f6b6301c925
I20260505 07:54:17.440634 2276 ts_tablet_manager.cc:1950] T 72da22d790624b07bfca6f6b6301c925 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.440946 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet bf76e868bbcb44b0b1887c84dd27f093 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]) successfully deleted
I20260505 07:54:17.441007 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 172) took 8389 ms. Trace:
I20260505 07:54:17.441078 2276 rpcz_store.cc:276] 0505 07:54:09.051191 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.051243 (+ 52us) service_pool.cc:224] Handling call
0505 07:54:17.413558 (+8362315us) ts_tablet_manager.cc:1151] Deleting tablet 72da22d790624b07bfca6f6b6301c925
0505 07:54:17.413568 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.426578 (+ 13010us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.440074 (+ 13496us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.440991 (+ 917us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8362138}
I20260505 07:54:17.441202 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 23fa401bb4634022bca3c6b23421e4d5 (table default.Table_0562bf620b5f46ecab060957ac1ca8bf [id=f47382441b814a62b5ff875ae632b915]) successfully deleted
I20260505 07:54:17.441350 2276 tablet_replica.cc:333] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.441540 2276 raft_consensus.cc:2243] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.441711 2276 raft_consensus.cc:2272] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.442267 2276 ts_tablet_manager.cc:1916] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.442884 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 3f1feb0387bc4afead72346398dbe9e5 (table default.Table_dcf56067c2654eff8773e49d0ddf5427 [id=a9ef91bfcff640abb23140249a59fcf4]) successfully deleted
I20260505 07:54:17.444453 2268 ts_tablet_manager.cc:1929] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.444613 2268 log.cc:1199] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/b256d072933b4566b5587224274deb66
I20260505 07:54:17.444975 2268 ts_tablet_manager.cc:1950] T b256d072933b4566b5587224274deb66 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.445634 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 231) took 6777 ms. Trace:
I20260505 07:54:17.445724 2268 rpcz_store.cc:276] 0505 07:54:10.667998 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.668441 (+ 443us) service_pool.cc:224] Handling call
0505 07:54:17.431482 (+6763041us) ts_tablet_manager.cc:1151] Deleting tablet b256d072933b4566b5587224274deb66
0505 07:54:17.431491 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.441577 (+ 10086us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.444413 (+ 2836us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.445618 (+ 1205us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6762616}
I20260505 07:54:17.446002 2268 tablet_replica.cc:333] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.446172 2268 raft_consensus.cc:2243] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.446309 2268 raft_consensus.cc:2272] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.446862 2447 ts_tablet_manager.cc:1916] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.447701 718 leader_election.cc:304] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:17.449777 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_5dd32763402445e9855bb9fe9c15dbdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.452344 990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260505 07:54:17.452507 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5dd32763402445e9855bb9fe9c15dbdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.460593 2276 ts_tablet_manager.cc:1929] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:17.460697 2276 log.cc:1199] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/f88a5c0d0e2c436c8d8b6afc1f3c2edf
I20260505 07:54:17.461077 2276 ts_tablet_manager.cc:1950] T f88a5c0d0e2c436c8d8b6afc1f3c2edf P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.461458 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 173) took 8389 ms. Trace:
I20260505 07:54:17.461544 2276 rpcz_store.cc:276] 0505 07:54:09.072303 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.072360 (+ 57us) service_pool.cc:224] Handling call
0505 07:54:17.441308 (+8368948us) ts_tablet_manager.cc:1151] Deleting tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf
0505 07:54:17.441318 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.454859 (+ 13541us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.460544 (+ 5685us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.461442 (+ 898us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8368759}
I20260505 07:54:17.461951 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 174) took 7768 ms. Trace:
I20260505 07:54:17.462028 2276 rpcz_store.cc:276] 0505 07:54:09.693782 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.693845 (+ 63us) service_pool.cc:224] Handling call
0505 07:54:17.461818 (+7767973us) ts_tablet_manager.cc:1151] Deleting tablet 05351965bdba4ccca8d13223cd505948
0505 07:54:17.461828 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.461933 (+ 105us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7767815}
I20260505 07:54:17.462311 2276 tablet_replica.cc:333] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.462508 2276 raft_consensus.cc:2243] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.464090 2268 ts_tablet_manager.cc:1916] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.469697 2268 ts_tablet_manager.cc:1929] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.469851 2268 log.cc:1199] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/df3a49b59ed6485f970074391f932482
I20260505 07:54:17.470209 2268 ts_tablet_manager.cc:1950] T df3a49b59ed6485f970074391f932482 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.462673 2276 raft_consensus.cc:2272] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.470981 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 230) took 6803 ms. Trace:
I20260505 07:54:17.471103 2268 rpcz_store.cc:276] 0505 07:54:10.667896 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.740637 (+ 72741us) service_pool.cc:224] Handling call
0505 07:54:17.445956 (+6705319us) ts_tablet_manager.cc:1151] Deleting tablet df3a49b59ed6485f970074391f932482
0505 07:54:17.445968 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.467072 (+ 21104us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.469660 (+ 2588us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.470962 (+ 1302us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6705154}
I20260505 07:54:17.471266 1348 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 83e62b7af9b847b7a9005e4345905c0d
I20260505 07:54:17.471692 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 233) took 6709 ms. Trace:
I20260505 07:54:17.471828 1348 catalog_manager.cc:6741] ReplaceTablet: tablet 83e62b7af9b847b7a9005e4345905c0d deleted and replaced by tablet a54d94d8a888483681858398f667a370
I20260505 07:54:17.471818 1172 tablet_bootstrap.cc:492] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.471776 2268 rpcz_store.cc:276] 0505 07:54:10.761982 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.762053 (+ 71us) service_pool.cc:224] Handling call
0505 07:54:17.471547 (+6709494us) ts_tablet_manager.cc:1151] Deleting tablet 91d57f5bb6e847e6b099134d1382b730
0505 07:54:17.471556 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.471677 (+ 121us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6709275}
I20260505 07:54:17.471952 1172 ts_tablet_manager.cc:1403] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.056s user 0.010s sys 0.008s
I20260505 07:54:17.472009 2268 tablet_replica.cc:333] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.472172 2268 raft_consensus.cc:2243] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.472306 2268 raft_consensus.cc:2272] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.472760 2268 ts_tablet_manager.cc:1916] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.472692 1172 raft_consensus.cc:359] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.472860 1172 raft_consensus.cc:385] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.472923 1172 raft_consensus.cc:740] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.473106 1172 consensus_queue.cc:260] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.473676 1172 ts_tablet_manager.cc:1434] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:17.473951 1172 tablet_bootstrap.cc:492] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.475564 1172 tablet_bootstrap.cc:654] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.475943 2276 ts_tablet_manager.cc:1916] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.487277 1172 tablet_bootstrap.cc:492] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.487416 1172 ts_tablet_manager.cc:1403] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.004s
I20260505 07:54:17.488507 1172 raft_consensus.cc:359] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.488716 1172 raft_consensus.cc:385] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.488780 1172 raft_consensus.cc:740] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.489037 1172 consensus_queue.cc:260] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.490800 2276 ts_tablet_manager.cc:1929] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.490911 2276 log.cc:1199] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/6fe19306f7df49fdb771d21f6560e697
I20260505 07:54:17.491322 2276 ts_tablet_manager.cc:1950] T 6fe19306f7df49fdb771d21f6560e697 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.491545 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet f1058fcb9fa6457a85489049ca1d9ada (table default.Table_2872bda04bf547648372a8f4bf5dc3ce [id=42ae40c0861e4489b724c9b8e3b590d7]) successfully deleted
I20260505 07:54:17.491694 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 175) took 7612 ms. Trace:
W20260505 07:54:17.491815 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet b12d2d2b866348d781209fa787da7d7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b12d2d2b866348d781209fa787da7d7e already in progress: creating tablet
I20260505 07:54:17.491775 2276 rpcz_store.cc:276] 0505 07:54:09.879376 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.879475 (+ 99us) service_pool.cc:224] Handling call
0505 07:54:17.462267 (+7582792us) ts_tablet_manager.cc:1151] Deleting tablet 6fe19306f7df49fdb771d21f6560e697
0505 07:54:17.462277 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.481948 (+ 19671us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.490754 (+ 8806us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.491675 (+ 921us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7582613}
I20260505 07:54:17.493441 1172 ts_tablet_manager.cc:1434] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260505 07:54:17.503297 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_02f83ad68c3f4615b60d1fc64f984e5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:17.503744 1349 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 99850198f4d04607b52953342daa753e
W20260505 07:54:17.503872 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_02f83ad68c3f4615b60d1fc64f984e5c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.504312 1349 catalog_manager.cc:6741] ReplaceTablet: tablet 99850198f4d04607b52953342daa753e deleted and replaced by tablet d73fd67f4b91479c8a53792f4f3779c2
I20260505 07:54:17.504769 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_19d6e7d3eebb4ebf9c35e22ba6a83ec3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.505355 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_19d6e7d3eebb4ebf9c35e22ba6a83ec3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.513515 1355 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f" } modify_external_catalogs: true
I20260505 07:54:17.513767 1355 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f" } modify_external_catalogs: true
W20260505 07:54:17.514161 990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260505 07:54:17.524135 2268 ts_tablet_manager.cc:1929] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.524246 2268 log.cc:1199] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/17b4e886bebc4ab6a3c95290d18b4229
I20260505 07:54:17.524566 2268 ts_tablet_manager.cc:1950] T 17b4e886bebc4ab6a3c95290d18b4229 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.525570 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 232) took 6763 ms. Trace:
I20260505 07:54:17.525679 2268 rpcz_store.cc:276] 0505 07:54:10.761786 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.762887 (+ 1101us) service_pool.cc:224] Handling call
0505 07:54:17.471971 (+6709084us) ts_tablet_manager.cc:1151] Deleting tablet 17b4e886bebc4ab6a3c95290d18b4229
0505 07:54:17.471979 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.475494 (+ 3515us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.524078 (+ 48584us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.525545 (+ 1467us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6708652}
I20260505 07:54:17.526144 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 234) took 6489 ms. Trace:
I20260505 07:54:17.526221 2268 rpcz_store.cc:276] 0505 07:54:11.036964 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.037624 (+ 660us) service_pool.cc:224] Handling call
0505 07:54:17.525997 (+6488373us) ts_tablet_manager.cc:1151] Deleting tablet 880e326cbea74233a0ec5abb31f5b0d7
0505 07:54:17.526011 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.526125 (+ 114us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6488204}
I20260505 07:54:17.526484 2268 tablet_replica.cc:333] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.526703 2268 raft_consensus.cc:2243] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.527022 2268 raft_consensus.cc:2272] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.527513 2268 ts_tablet_manager.cc:1916] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.528720 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_0f05cd371317454b9be3fe9d7c31a9c4" } new_table_name: "default.Table_4e651b0d817f408e8e36b0399a57a04f" modify_external_catalogs: true
I20260505 07:54:17.534759 2268 ts_tablet_manager.cc:1929] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.536784 2268 log.cc:1199] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/abe5f6f0774545b296bf1b03a08e145d
I20260505 07:54:17.537205 2268 ts_tablet_manager.cc:1950] T abe5f6f0774545b296bf1b03a08e145d P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.537868 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 245) took 6349 ms. Trace:
I20260505 07:54:17.537984 2268 rpcz_store.cc:276] 0505 07:54:11.188244 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.188294 (+ 50us) service_pool.cc:224] Handling call
0505 07:54:17.526438 (+6338144us) ts_tablet_manager.cc:1151] Deleting tablet abe5f6f0774545b296bf1b03a08e145d
0505 07:54:17.526447 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.531132 (+ 4685us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.534706 (+ 3574us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.537852 (+ 3146us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6337902}
I20260505 07:54:17.538295 2268 tablet_replica.cc:333] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.538483 2268 raft_consensus.cc:2243] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.538729 2268 raft_consensus.cc:2272] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.539243 2268 ts_tablet_manager.cc:1916] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.539944 1172 tablet_bootstrap.cc:492] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.541467 1172 tablet_bootstrap.cc:654] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.545750 2276 tablet_replica.cc:333] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.545758 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet d3d11bd19f074bbab101448b45426b7a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]) successfully deleted
I20260505 07:54:17.552448 2268 ts_tablet_manager.cc:1929] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.563671 2268 log.cc:1199] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/2d86b8b4dadf4fd2b6409a8cb70e313e
I20260505 07:54:17.564113 2268 ts_tablet_manager.cc:1950] T 2d86b8b4dadf4fd2b6409a8cb70e313e P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.565554 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 246) took 6376 ms. Trace:
I20260505 07:54:17.565640 2268 rpcz_store.cc:276] 0505 07:54:11.188767 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.188826 (+ 59us) service_pool.cc:224] Handling call
0505 07:54:17.538258 (+6349432us) ts_tablet_manager.cc:1151] Deleting tablet 2d86b8b4dadf4fd2b6409a8cb70e313e
0505 07:54:17.538266 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.548832 (+ 10566us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.552394 (+ 3562us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.565537 (+ 13143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":677,"tablet-delete.queue_time_us":6349269}
I20260505 07:54:17.565950 2268 tablet_replica.cc:333] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.566124 2268 raft_consensus.cc:2243] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.452894 2447 ts_tablet_manager.cc:1929] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.487653 744 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c4fc18f6faa488882d9bffb30793cdd with delete_type TABLET_DATA_DELETED (replaced by tablet 57b5378ec4ae40448311d54c79ea33b0) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.489091 749 tablet_service.cc:1558] Processing DeleteTablet for tablet 83e62b7af9b847b7a9005e4345905c0d with delete_type TABLET_DATA_DELETED (replaced by tablet a54d94d8a888483681858398f667a370) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.566329 2447 log.cc:1199] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/09b46ff0101a4e529077b9e30492ace7
I20260505 07:54:17.542114 762 tablet_service.cc:1511] Processing CreateTablet for tablet b53db8fdadad4ec09d05f65c84100ff0 (DEFAULT_TABLE table=default.Table_d17cd5686eb94c3b8993d4310f0d31f7 [id=8c0e89be595c4eb2b4d4103121cdccf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:17.566685 2447 ts_tablet_manager.cc:1950] T 09b46ff0101a4e529077b9e30492ace7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:17.566782 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b53db8fdadad4ec09d05f65c84100ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.566993 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 170) took 8016 ms. Trace:
I20260505 07:54:17.567060 2447 rpcz_store.cc:276] 0505 07:54:09.550372 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.550418 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:17.437476 (+7887058us) ts_tablet_manager.cc:1151] Deleting tablet 09b46ff0101a4e529077b9e30492ace7
0505 07:54:17.437487 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.450088 (+ 12601us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.452843 (+ 2755us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.566977 (+114134us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7878378}
I20260505 07:54:17.567329 2447 tablet_replica.cc:333] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:17.569144 1172 tablet_bootstrap.cc:492] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.569290 1172 ts_tablet_manager.cc:1403] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.005s
I20260505 07:54:17.569968 1172 raft_consensus.cc:359] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.570138 1172 raft_consensus.cc:385] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.570195 1172 raft_consensus.cc:740] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.543313 756 tablet_service.cc:1511] Processing CreateTablet for tablet 27ab0ebc13f349f38cf86b9643700ae1 (DEFAULT_TABLE table=default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:17.570405 1172 consensus_queue.cc:260] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.570555 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27ab0ebc13f349f38cf86b9643700ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.570871 1172 ts_tablet_manager.cc:1434] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:17.571115 1172 tablet_bootstrap.cc:492] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.572666 1172 tablet_bootstrap.cc:654] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.543813 745 tablet_service.cc:1558] Processing DeleteTablet for tablet 99850198f4d04607b52953342daa753e with delete_type TABLET_DATA_DELETED (replaced by tablet d73fd67f4b91479c8a53792f4f3779c2) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.582528 2268 raft_consensus.cc:2272] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.583041 2778 raft_consensus.cc:493] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.583257 2268 ts_tablet_manager.cc:1916] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.583204 2778 raft_consensus.cc:515] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.583451 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:17.583874 2778 leader_election.cc:290] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:17.584475 1172 tablet_bootstrap.cc:492] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.584633 1172 ts_tablet_manager.cc:1403] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.002s
I20260505 07:54:17.584756 2872 raft_consensus.cc:2749] T 2706bc9b9e02414693062deff5cfe81b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:17.585036 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 137359us
I20260505 07:54:17.585352 1172 raft_consensus.cc:359] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.585498 1172 raft_consensus.cc:385] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.585547 1172 raft_consensus.cc:740] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.585740 1172 consensus_queue.cc:260] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.586100 1172 ts_tablet_manager.cc:1434] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260505 07:54:17.545185 717 consensus_peers.cc:597] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8c9eaaef250461ca7a37d0cf42b07ee. This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:17.586345 1172 tablet_bootstrap.cc:492] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.588052 1172 tablet_bootstrap.cc:654] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:17.589507 856 leader_election.cc:343] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:17.590142 2268 ts_tablet_manager.cc:1929] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.590315 2268 log.cc:1199] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/d7e0c8ad716e443588d4ae3aa28ed4bf
I20260505 07:54:17.590283 1878 raft_consensus.cc:493] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.590492 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:17.590499 1878 raft_consensus.cc:515] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.590839 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:17.590894 2268 ts_tablet_manager.cc:1950] T d7e0c8ad716e443588d4ae3aa28ed4bf P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.591140 1878 leader_election.cc:290] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:17.591282 1354 catalog_manager.cc:5671] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:17.591760 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:17.591974 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:17.592118 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:17.592325 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet c837877946604ef29f95dac9dd449bf3 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]). Expected version 1 got 0
I20260505 07:54:17.592567 2276 raft_consensus.cc:2243] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.592684 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:17.592788 2276 raft_consensus.cc:2272] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.593111 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:17.593367 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:17.593385 2276 ts_tablet_manager.cc:1916] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.593546 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:17.593711 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:17.593894 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:17.594089 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 423c22891bed462d89e9ec51a079f0f5 (table default.Table_44c3bcc031ad40a796ea43c44863d46f [id=b8e0010041cc477d9ce226cca5a28a52]). Expected version 4 got 0
I20260505 07:54:17.594674 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:17.595263 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:17.595774 3145 raft_consensus.cc:493] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.595911 3145 raft_consensus.cc:515] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.596539 3145 leader_election.cc:290] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:17.597136 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad5c570d4861460d8ab0c8ff8a7a3a03" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:17.597252 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:17.597558 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
W20260505 07:54:17.597615 854 leader_election.cc:343] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:17.597862 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:17.598486 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:17.599063 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_65d14038399640c284b8382aa80e344f [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 2 got 0
I20260505 07:54:17.599366 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:17.599601 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 490483562d8f4c2daa7eaa333a9f3a0c (table default.Table_3acf7c3eb13d473a9d952098c7524753 [id=fe7ee6e8118840e8a9f70a007d80c2b3]). Expected version 1 got 0
I20260505 07:54:17.599779 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet e38f248556bd4e49b06da72a1cbd7d8f (table default.Table_5c2b133923064d5eb9eabc21fb62f098 [id=12443115e63e4428a9cb798980fcbf5b]). Expected version 1 got 0
I20260505 07:54:17.600102 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:17.600502 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:17.600855 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1506cd31fe364e328867bbf70e36845e (table default.Table_eee3bd15017341fca226f671948e94e9 [id=c7cad0401e40485e9ed03160b9bed704]). Expected version 1 got 0
I20260505 07:54:17.601032 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:17.601330 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:17.601925 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:17.602170 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:17.602641 2276 ts_tablet_manager.cc:1929] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:17.602751 2276 log.cc:1199] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/5683d7366de84653ace56f0edcd3818c
I20260505 07:54:17.602764 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:17.603097 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:17.603133 2276 ts_tablet_manager.cc:1950] T 5683d7366de84653ace56f0edcd3818c P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.603899 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 177) took 7251 ms. Trace:
I20260505 07:54:17.603926 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:17.603986 2276 rpcz_store.cc:276] 0505 07:54:10.352532 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.352599 (+ 67us) service_pool.cc:224] Handling call
0505 07:54:17.492046 (+7139447us) ts_tablet_manager.cc:1151] Deleting tablet 5683d7366de84653ace56f0edcd3818c
0505 07:54:17.539690 (+ 47644us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.599168 (+ 59478us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.602601 (+ 3433us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.603877 (+ 1276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46616,"tablet-delete.queue_time_us":7139289}
I20260505 07:54:17.604265 2276 tablet_replica.cc:333] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.604247 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 90612f0afb4949f491e625c61e40d247 (table default.Table_61a6fc82edd5408f991f40a8d15394a5 [id=9930805fdf364f25aea11f859b50bc71]). Expected version 1 got 0
I20260505 07:54:17.604395 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:17.604426 2276 raft_consensus.cc:2243] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.604545 2276 raft_consensus.cc:2272] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.605057 2276 ts_tablet_manager.cc:1916] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.604883 1354 catalog_manager.cc:5671] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:17.605468 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:17.606078 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:17.606344 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:17.606843 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:17.607046 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:17.607252 1354 catalog_manager.cc:5671] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:17.607751 2872 consensus_queue.cc:1048] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.610673 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 09b46ff0101a4e529077b9e30492ace7 (table default.Table_c12be1928760438fab7c135df431bc79 [id=d567651314004cf4b33a8dfc1ef30698]) successfully deleted
I20260505 07:54:17.611634 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 247) took 6422 ms. Trace:
I20260505 07:54:17.611717 2268 rpcz_store.cc:276] 0505 07:54:11.189261 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.189306 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:17.565912 (+6376606us) ts_tablet_manager.cc:1151] Deleting tablet d7e0c8ad716e443588d4ae3aa28ed4bf
0505 07:54:17.565921 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.586313 (+ 20392us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.590099 (+ 3786us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.611614 (+ 21515us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36049,"tablet-delete.queue_time_us":6376447}
I20260505 07:54:17.612004 2268 tablet_replica.cc:333] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.612169 2268 raft_consensus.cc:2243] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.592571 856 leader_election.cc:304] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:17.612296 2268 raft_consensus.cc:2272] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.616446 3145 raft_consensus.cc:2749] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:17.616552 2268 ts_tablet_manager.cc:1916] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.621444 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:17.622143 2268 ts_tablet_manager.cc:1929] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.622181 1354 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:17.622238 2268 log.cc:1199] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/ba616702dc6040bb805bb3ffc8a8a871
I20260505 07:54:17.622550 2268 ts_tablet_manager.cc:1950] T ba616702dc6040bb805bb3ffc8a8a871 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.623208 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 248) took 6433 ms. Trace:
I20260505 07:54:17.623291 2268 rpcz_store.cc:276] 0505 07:54:11.189667 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.189713 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:17.611962 (+6422249us) ts_tablet_manager.cc:1151] Deleting tablet ba616702dc6040bb805bb3ffc8a8a871
0505 07:54:17.611972 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.619423 (+ 7451us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.622101 (+ 2678us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.623190 (+ 1089us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6422081}
I20260505 07:54:17.623935 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 249) took 6433 ms. Trace:
I20260505 07:54:17.624054 2268 rpcz_store.cc:276] 0505 07:54:11.190035 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.190075 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:17.623512 (+6433437us) ts_tablet_manager.cc:1151] Deleting tablet accb2dedc30e494ebd22c33594cee661
0505 07:54:17.623522 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.623919 (+ 397us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6433268}
I20260505 07:54:17.624341 2268 tablet_replica.cc:333] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.624555 2268 raft_consensus.cc:2243] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.624758 2268 raft_consensus.cc:2272] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.625281 2268 ts_tablet_manager.cc:1916] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:17.631484 855 connection.cc:570] client connection to 127.0.105.60:35401 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260505 07:54:17.631505 1317 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.0.105.60:35401 (error 108) [suppressed 10 similar messages]
W20260505 07:54:17.615060 718 leader_election.cc:343] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: a0f2f46d0e7840458cb992ce2a2dbea2
I20260505 07:54:17.631884 718 leader_election.cc:304] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
W20260505 07:54:17.631914 716 connection.cc:570] client connection to 127.0.105.60:35401 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260505 07:54:17.632068 989 connection.cc:570] client connection to 127.0.105.60:35401 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260505 07:54:17.632660 718 consensus_peers.cc:597] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8c9eaaef250461ca7a37d0cf42b07ee. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:17.633452 718 leader_election.cc:304] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:17.633845 3013 raft_consensus.cc:2749] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:17.633930 990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260505 07:54:17.633987 3133 consensus_queue.cc:1048] T ec145bdc0dab4642ba233bd0da2c436f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.634198 3013 raft_consensus.cc:2749] T abaa596871bb4a82aefc317932e91d35 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:17.634634 3133 consensus_queue.cc:1048] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.636150 718 leader_election.cc:304] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:17.636469 3133 raft_consensus.cc:2804] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:17.636548 3133 raft_consensus.cc:493] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:17.636623 3133 raft_consensus.cc:3060] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:17.639883 3133 raft_consensus.cc:515] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.640508 3133 leader_election.cc:290] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:17.641340 3133 consensus_queue.cc:1048] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.644259 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef22101fe014e21adea6e10f4b550e7" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.644470 907 raft_consensus.cc:2468] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:17.645567 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4ef22101fe014e21adea6e10f4b550e7" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:17.645753 907 raft_consensus.cc:3060] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:17.614799 991 leader_election.cc:304] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:17.648931 907 raft_consensus.cc:2468] T 4ef22101fe014e21adea6e10f4b550e7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:17.649125 1878 raft_consensus.cc:2764] T 4ef22101fe014e21adea6e10f4b550e7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260505 07:54:17.651921 991 leader_election.cc:304] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:17.652194 1878 raft_consensus.cc:2804] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:17.652328 1878 raft_consensus.cc:697] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:17.652652 1878 consensus_queue.cc:237] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.657557 3100 tablet.cc:1724] T f84f464e83cb412eade9978ea99f6685 P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:17.662108 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd2e2514c72419390c1bb9bd13f1fb3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.662388 921 raft_consensus.cc:2468] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:17.663193 921 raft_consensus.cc:3060] T b4515a0ff8304c69b9aa030e9b14d43f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:17.664574 3004 raft_consensus.cc:493] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.665156 3004 raft_consensus.cc:515] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.666725 3004 leader_election.cc:290] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:17.670894 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a82ed2b87c487aac84194f2dcb0d6b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:17.671073 913 raft_consensus.cc:3060] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:17.673784 913 raft_consensus.cc:2468] T 29a82ed2b87c487aac84194f2dcb0d6b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:17.676204 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbfb2345a1a4b218c2dddbae6667e47" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:17.678681 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:17.679282 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7887c3462f74740b314e3bec36e280d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:17.679527 772 raft_consensus.cc:2468] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:17.680660 1352 catalog_manager.cc:5671] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:17.691824 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:17.692263 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:17.692607 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:17.693140 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:17.693691 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:17.693810 776 raft_consensus.cc:3060] T 4b528d6c36ac4920b6ea0a6e26cca269 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:17.694026 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:17.694339 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:17.695003 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:17.695714 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:17.696105 3112 tablet.cc:1724] T 66082a7a75e345639e7a268727922fbb P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:17.696303 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:17.696605 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:17.697510 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:17.697790 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 490483562d8f4c2daa7eaa333a9f3a0c (table default.Table_3acf7c3eb13d473a9d952098c7524753 [id=fe7ee6e8118840e8a9f70a007d80c2b3]). Expected version 1 got 0
I20260505 07:54:17.698315 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:17.699546 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_65d14038399640c284b8382aa80e344f [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 2 got 0
I20260505 07:54:17.705329 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82e9ae417784b7e83f4843b129d0580" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:17.707887 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a890d5e3f1af49b5aaa28c6c51cb3874" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:17.708102 781 raft_consensus.cc:2468] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:17.708595 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:17.709110 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:17.709437 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet e38f248556bd4e49b06da72a1cbd7d8f (table default.Table_5c2b133923064d5eb9eabc21fb62f098 [id=12443115e63e4428a9cb798980fcbf5b]). Expected version 1 got 0
I20260505 07:54:17.709965 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:17.710508 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_db06b2a6ada74addb6f217ab26d73870 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 1 got 0
I20260505 07:54:17.710999 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:17.711264 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1506cd31fe364e328867bbf70e36845e (table default.Table_eee3bd15017341fca226f671948e94e9 [id=c7cad0401e40485e9ed03160b9bed704]). Expected version 1 got 0
I20260505 07:54:17.711501 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:17.712011 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:17.712522 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 01431ad112e8445487823efe8feb0dbe (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:17.713465 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:17.714726 1352 catalog_manager.cc:5671] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:17.717815 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:17.717964 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a890d5e3f1af49b5aaa28c6c51cb3874" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.718190 921 raft_consensus.cc:2468] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:17.718227 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:17.719332 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 429ccdb56f0b456483cf1debb5de0beb (table default.Table_6235f4345c354a6daa96d2ece798f82e [id=1a0d30f064af4c2f8e082d5742ba6d28]). Expected version 1 got 0
I20260505 07:54:17.719684 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:17.720079 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 90612f0afb4949f491e625c61e40d247 (table default.Table_61a6fc82edd5408f991f40a8d15394a5 [id=9930805fdf364f25aea11f859b50bc71]). Expected version 1 got 0
I20260505 07:54:17.720690 1352 catalog_manager.cc:5671] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:17.721330 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:17.721751 1352 catalog_manager.cc:5671] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:17.722420 1352 catalog_manager.cc:5671] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:17.723148 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:17.723439 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:17.723754 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:17.724525 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:17.725032 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:17.725607 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:17.725965 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:17.726337 1352 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:17.727275 1351 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 29a82ed2b87c487aac84194f2dcb0d6b
I20260505 07:54:17.727945 1351 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 218) took 1290 ms. Trace:
I20260505 07:54:17.728066 1351 rpcz_store.cc:276] 0505 07:54:16.437898 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.087802 (+649904us) service_pool.cc:224] Handling call
0505 07:54:17.216922 (+129120us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d7b06 00007fd65a46c486 00007fd65a46c1b5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:17.217261 (+ 339us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:17.217388 (+ 127us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:17.217456 (+ 68us) catalog_manager.cc:2824] Locking tablets
0505 07:54:17.217601 (+ 145us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:17.408342 (+190741us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:17.434971 (+ 26629us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:17.727218 (+292247us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:17.727874 (+ 656us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.269604 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.269704 (+ 100us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.269709 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:17.269711 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.269794 (+ 83us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.269802 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.269807 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.269898 (+ 91us) tablet.cc:701] Row locks acquired
0505 07:54:17.269900 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.269927 (+ 27us) write_op.cc:270] Start()
0505 07:54:17.269952 (+ 25us) write_op.cc:276] Timestamp: P: 1777967657269871 usec, L: 0
0505 07:54:17.269956 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.270095 (+ 139us) log.cc:844] Serialized 725 byte log entry
0505 07:54:17.404107 (+134012us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.406714 (+ 2607us) write_op.cc:301] APPLY: starting
0505 07:54:17.406723 (+ 9us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.406751 (+ 28us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.406770 (+ 19us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.406822 (+ 52us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.406843 (+ 21us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:17.406850 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:17.406894 (+ 44us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.406926 (+ 32us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.406937 (+ 11us) write_op.cc:454] Released schema lock
0505 07:54:17.406942 (+ 5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":289730944,"child_traces":[["op",{"apply.queue_time_us":2581,"apply.run_cpu_time_us":273,"apply.run_wall_time_us":273,"num_ops":2,"prepare.queue_time_us":51332,"prepare.run_cpu_time_us":561,"prepare.run_wall_time_us":560,"replication_time_us":134138,"thread_start_us":45216,"threads_started":1}]]}
I20260505 07:54:17.729012 1351 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_b85a2b107c88498e84217252ca6cb947" } modify_external_catalogs: true
I20260505 07:54:17.729177 1351 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_b85a2b107c88498e84217252ca6cb947" } modify_external_catalogs: true
I20260505 07:54:17.727108 2872 raft_consensus.cc:493] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:17.730885 2872 raft_consensus.cc:515] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:17.731464 3185 tablet.cc:1724] T 423c22891bed462d89e9ec51a079f0f5 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 4
I20260505 07:54:17.731657 2872 leader_election.cc:290] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:17.732596 3186 tablet.cc:1724] T c837877946604ef29f95dac9dd449bf3 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260505 07:54:17.736788 1353 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e243711164f04faabbde53b0a09d90b1 consensus state has no leader
W20260505 07:54:17.737200 1350 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d86214559d014f028acc5a9691efc9ca consensus state has no leader
I20260505 07:54:17.737850 1355 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet dd3fc8a9d9f04636b1f31bb88411c1da
I20260505 07:54:17.737960 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 29a82ed2b87c487aac84194f2dcb0d6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.740255 1355 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_2634c105c2c041959e881249dddecdaf" } modify_external_catalogs: true
W20260505 07:54:17.741343 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1bd5951bfc3346c2a6aad8f33357394a consensus state has no leader
W20260505 07:54:17.743549 1317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.60:35401, user_credentials={real_user=slave}} blocked reactor thread for 112019us
I20260505 07:54:17.749622 3144 raft_consensus.cc:493] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.749788 3144 raft_consensus.cc:515] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.750303 3144 leader_election.cc:290] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:17.750942 856 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 131233us
W20260505 07:54:17.751245 856 consensus_peers.cc:597] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 09b46ff0101a4e529077b9e30492ace7. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:17.752885 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 25072055f5534078adf5b5025614f951 (table default.Table_f93a81c5dfe740819f0f6ea5bf7f1067 [id=a8a6d42a6b6748d590ef5949a9d1e650]) successfully deleted
I20260505 07:54:17.753439 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 72da22d790624b07bfca6f6b6301c925 (table default.Table_201f8409684e49be8bc5cd5249606250 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1]) successfully deleted
I20260505 07:54:17.753937 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf (table default.Table_fdb48e51d96448b7ab7ff0d7b7c31d1c [id=8a910f6a553a47ea87ec170e376982d1]) successfully deleted
W20260505 07:54:17.754391 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 05351965bdba4ccca8d13223cd505948 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05351965bdba4ccca8d13223cd505948 already in progress: creating tablet
I20260505 07:54:17.754840 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 6fe19306f7df49fdb771d21f6560e697 (table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]) successfully deleted
I20260505 07:54:17.755602 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 5683d7366de84653ace56f0edcd3818c (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]) successfully deleted
I20260505 07:54:17.758479 3149 raft_consensus.cc:493] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.758625 3149 raft_consensus.cc:515] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.759316 3149 leader_election.cc:290] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:17.760831 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b48208e68247e78c12f0fec7ba07dc" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:17.761425 854 leader_election.cc:343] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:17.762188 717 consensus_peers.cc:597] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5683d7366de84653ace56f0edcd3818c. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:17.762573 717 consensus_peers.cc:597] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9959be9f83c2444282d012b61eef49b5. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:17.762804 717 consensus_peers.cc:597] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b5a0cbf9c3d74cea92920189da332c1c. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:17.763432 717 consensus_peers.cc:597] T 54e4f7f895d943639653c8c32724baf2 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:17.764379 717 leader_election.cc:343] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 02412333633a4cdebe95a144fa6c363c
I20260505 07:54:17.764873 756 tablet_service.cc:1558] Processing DeleteTablet for tablet dd3fc8a9d9f04636b1f31bb88411c1da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.766562 1176 tablet_bootstrap.cc:492] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:17.766772 1176 ts_tablet_manager.cc:1403] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.353s user 0.004s sys 0.006s
I20260505 07:54:17.767748 1176 raft_consensus.cc:359] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.768049 1176 raft_consensus.cc:385] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.768167 1176 raft_consensus.cc:740] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.768427 1176 consensus_queue.cc:260] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.769081 1176 ts_tablet_manager.cc:1434] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:17.770572 1172 tablet_bootstrap.cc:492] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.770730 1172 ts_tablet_manager.cc:1403] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.184s user 0.007s sys 0.003s
I20260505 07:54:17.770833 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f069982138460da3b7ae7a0b4e3833" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:17.771425 854 leader_election.cc:343] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:17.771651 1172 raft_consensus.cc:359] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.771844 1172 raft_consensus.cc:385] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.771910 1172 raft_consensus.cc:740] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.772102 1172 consensus_queue.cc:260] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.772651 1172 ts_tablet_manager.cc:1434] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:17.772950 1172 tablet_bootstrap.cc:492] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:17.729159 718 consensus_peers.cc:597] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b57c1c618484d5d86a846fb982a2240. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:17.774852 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 45716.3us
I20260505 07:54:17.775148 1172 tablet_bootstrap.cc:654] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:17.775410 718 consensus_peers.cc:597] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0baf83b155494fddb3c25144608706f7. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:17.775658 718 leader_election.cc:304] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:17.776213 2872 consensus_queue.cc:1048] T 4b9963a0f1a54ee9a73486bb1801ee74 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.776850 2872 raft_consensus.cc:2804] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:17.777012 2872 raft_consensus.cc:493] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:17.777148 2872 raft_consensus.cc:3060] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:17.779656 3161 consensus_queue.cc:1048] T b4515a0ff8304c69b9aa030e9b14d43f P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.789564 1172 tablet_bootstrap.cc:492] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.789741 1172 ts_tablet_manager.cc:1403] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.000s
I20260505 07:54:17.790469 1172 raft_consensus.cc:359] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.790618 1172 raft_consensus.cc:385] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.790707 1172 raft_consensus.cc:740] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.790902 1172 consensus_queue.cc:260] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.791356 1172 ts_tablet_manager.cc:1434] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260505 07:54:17.791682 1172 tablet_bootstrap.cc:492] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.796914 1172 tablet_bootstrap.cc:654] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.797592 2872 raft_consensus.cc:515] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.801859 3194 tablet.cc:1724] T 429ccdb56f0b456483cf1debb5de0beb P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:17.802651 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c4fc18f6faa488882d9bffb30793cdd with delete_type TABLET_DATA_DELETED (replaced by tablet 57b5378ec4ae40448311d54c79ea33b0) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.803107 887 tablet_service.cc:1558] Processing DeleteTablet for tablet 83e62b7af9b847b7a9005e4345905c0d with delete_type TABLET_DATA_DELETED (replaced by tablet a54d94d8a888483681858398f667a370) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.803714 887 tablet_service.cc:1511] Processing CreateTablet for tablet b53db8fdadad4ec09d05f65c84100ff0 (DEFAULT_TABLE table=default.Table_d17cd5686eb94c3b8993d4310f0d31f7 [id=8c0e89be595c4eb2b4d4103121cdccf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:17.803985 896 tablet_service.cc:1558] Processing DeleteTablet for tablet dd3fc8a9d9f04636b1f31bb88411c1da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.804059 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b53db8fdadad4ec09d05f65c84100ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.807952 885 tablet_service.cc:1511] Processing CreateTablet for tablet 27ab0ebc13f349f38cf86b9643700ae1 (DEFAULT_TABLE table=default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:17.808320 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27ab0ebc13f349f38cf86b9643700ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.808619 3004 consensus_queue.cc:1048] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.809017 899 tablet_service.cc:1558] Processing DeleteTablet for tablet 29a82ed2b87c487aac84194f2dcb0d6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:17.811771 718 consensus_peers.cc:597] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d77ff90ec2ff4b6eb1795d149459c23a. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:17.812039 718 leader_election.cc:304] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:17.812247 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 99850198f4d04607b52953342daa753e with delete_type TABLET_DATA_DELETED (replaced by tablet d73fd67f4b91479c8a53792f4f3779c2) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:17.812422 3004 raft_consensus.cc:2804] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:17.812589 3004 raft_consensus.cc:697] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
W20260505 07:54:17.818276 1317 consensus_peers.cc:597] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 -> Peer 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401): Couldn't send request to peer 28d3eda04f9c47ae976d19c218b6dcc5. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:17.825603 1354 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_d99f0c6575024655ab3e0c737bc16910 [id=646a36b5d2b64ae3836951b29ae9ac00] alter complete (version 1)
W20260505 07:54:17.831259 717 consensus_peers.cc:597] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df3a49b59ed6485f970074391f932482. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:17.831575 1358 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_03da93b15bba4cea9230ec39dfa58b5b [id=c10b484439c443368141ceb4cf43365f] alter complete (version 1)
W20260505 07:54:17.832603 718 consensus_peers.cc:597] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:17.833636 718 leader_election.cc:304] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:17.848737 1353 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_d798bdca20804d6eb6aa52b61c2b69e3" } new_table_name: "default.Table_115e3886a51546409c5e744efd8b0462" modify_external_catalogs: true
I20260505 07:54:17.850124 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_e70b90b3e24449e3953b4a25068a50e5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:17.850314 2276 ts_tablet_manager.cc:1929] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:17.850406 2276 log.cc:1199] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/df3a49b59ed6485f970074391f932482
W20260505 07:54:17.850636 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e70b90b3e24449e3953b4a25068a50e5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.850677 2276 ts_tablet_manager.cc:1950] T df3a49b59ed6485f970074391f932482 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:17.852232 1355 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_2634c105c2c041959e881249dddecdaf" } modify_external_catalogs: true
I20260505 07:54:17.852907 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_921c2024ffef4029a90fe927e10bb705"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.853384 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_921c2024ffef4029a90fe927e10bb705 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.856478 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 189) took 7195 ms. Trace:
W20260505 07:54:17.856762 1358 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.1:36755 (request call id 148) took 1807 ms (client timeout 1000 ms). Trace:
W20260505 07:54:17.856881 1358 rpcz_store.cc:269] 0505 07:54:16.049340 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.230696 (+181356us) service_pool.cc:224] Handling call
0505 07:54:16.851555 (+620859us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.851560 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:17.856728 (+1005168us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.844602 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:16.844756 (+ 154us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:16.844763 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:16.844767 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:16.845010 (+ 243us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:16.845027 (+ 17us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:16.845035 (+ 8us) tablet.cc:685] Acquiring locks for 6 operations
0505 07:54:16.845246 (+ 211us) tablet.cc:701] Row locks acquired
0505 07:54:16.845249 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:16.845287 (+ 38us) write_op.cc:270] Start()
0505 07:54:16.845323 (+ 36us) write_op.cc:276] Timestamp: P: 1777967656845212 usec, L: 0
0505 07:54:16.845326 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.845502 (+ 176us) log.cc:844] Serialized 2398 byte log entry
0505 07:54:16.849116 (+ 3614us) op_driver.cc:464] REPLICATION: finished
0505 07:54:16.849229 (+ 113us) write_op.cc:301] APPLY: starting
0505 07:54:16.849243 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:16.849294 (+ 51us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:16.849328 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:16.849454 (+ 126us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:16.849492 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0505 07:54:16.849498 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:16.849583 (+ 85us) log.cc:844] Serialized 53 byte log entry
0505 07:54:16.849657 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:16.849683 (+ 26us) write_op.cc:454] Released schema lock
0505 07:54:16.849690 (+ 7us) write_op.cc:341] FINISH: Updating metrics
Related trace 'op':
0505 07:54:17.744078 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.744198 (+ 120us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.744204 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:17.744206 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.744326 (+ 120us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.744338 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.744343 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:17.744406 (+ 63us) tablet.cc:701] Row locks acquired
0505 07:54:17.744408 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.744436 (+ 28us) write_op.cc:270] Start()
0505 07:54:17.744468 (+ 32us) write_op.cc:276] Timestamp: P: 1777967657744383 usec, L: 0
0505 07:54:17.744472 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.744640 (+ 168us) log.cc:844] Serialized 356 byte log entry
0505 07:54:17.824036 (+ 79396us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.824385 (+ 349us) write_op.cc:301] APPLY: starting
0505 07:54:17.824405 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.824448 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.824481 (+ 33us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.824577 (+ 96us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.824623 (+ 46us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:17.824635 (+ 12us) write_op.cc:312] APPLY: finished
0505 07:54:17.824712 (+ 77us) log.cc:844] Serialized 23 byte log entry
0505 07:54:17.824772 (+ 60us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.824792 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:17.824804 (+ 12us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.824888 (+ 84us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.824894 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":78013,"reported_tablets":274,"child_traces":[["op",{"apply.queue_time_us":67,"apply.run_cpu_time_us":513,"apply.run_wall_time_us":512,"num_ops":6,"prepare.queue_time_us":1401,"prepare.run_cpu_time_us":1012,"prepare.run_wall_time_us":1011,"raft.queue_time_us":404,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":237,"replication_time_us":3763}],["op",{"apply.queue_time_us":301,"apply.run_cpu_time_us":494,"apply.run_wall_time_us":493,"num_ops":1,"prepare.queue_time_us":119,"prepare.run_cpu_time_us":627,"prepare.run_wall_time_us":931,"replication_time_us":79548}]]}
I20260505 07:54:17.858456 2276 rpcz_store.cc:276] 0505 07:54:10.661105 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.661298 (+ 193us) service_pool.cc:224] Handling call
0505 07:54:17.604226 (+6942928us) ts_tablet_manager.cc:1151] Deleting tablet df3a49b59ed6485f970074391f932482
0505 07:54:17.604234 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.798417 (+194183us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.850261 (+ 51844us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.856460 (+ 6199us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6942781}
I20260505 07:54:17.863212 3149 raft_consensus.cc:493] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.863349 3149 raft_consensus.cc:515] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.863785 1358 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_7522f51dac38424194a5755cf5b39067"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:17.863965 3149 leader_election.cc:290] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:17.864271 1358 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7522f51dac38424194a5755cf5b39067 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.864969 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet df3a49b59ed6485f970074391f932482 (table default.Table_7af623f581af498b86c8c3b1adbc6b80 [id=306149ae47a745edabf863d5004d5012]) successfully deleted
I20260505 07:54:17.865197 2276 tablet_replica.cc:333] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:17.865379 2276 raft_consensus.cc:2243] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.865733 2276 raft_consensus.cc:2272] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.866227 2276 ts_tablet_manager.cc:1916] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.868839 1172 tablet_bootstrap.cc:492] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.869019 1172 ts_tablet_manager.cc:1403] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.077s user 0.012s sys 0.029s
I20260505 07:54:17.869769 1172 raft_consensus.cc:359] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.869984 1172 raft_consensus.cc:385] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.870078 1172 raft_consensus.cc:740] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.870325 1172 consensus_queue.cc:260] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.871467 1172 ts_tablet_manager.cc:1434] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:17.875906 1172 tablet_bootstrap.cc:492] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.877478 1172 tablet_bootstrap.cc:654] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.746696 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29a82ed2b87c487aac84194f2dcb0d6b" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:17.889689 1053 raft_consensus.cc:3060] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:17.892940 1053 raft_consensus.cc:2468] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:17.746984 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a890d5e3f1af49b5aaa28c6c51cb3874" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:17.893492 1050 raft_consensus.cc:2468] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:17.811506 2268 ts_tablet_manager.cc:1929] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.893711 2268 log.cc:1199] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/54e4f7f895d943639653c8c32724baf2
I20260505 07:54:17.894079 2268 ts_tablet_manager.cc:1950] T 54e4f7f895d943639653c8c32724baf2 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
W20260505 07:54:17.894343 3100 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.236s user 0.004s sys 0.000s
I20260505 07:54:17.894917 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 250) took 6704 ms. Trace:
I20260505 07:54:17.895005 2268 rpcz_store.cc:276] 0505 07:54:11.190419 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.190459 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:17.624302 (+6433843us) ts_tablet_manager.cc:1151] Deleting tablet 54e4f7f895d943639653c8c32724baf2
0505 07:54:17.624310 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.798589 (+174279us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.811458 (+ 12869us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.894900 (+ 83442us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6433719}
I20260505 07:54:17.895422 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 244) took 6707 ms. Trace:
I20260505 07:54:17.895500 2268 rpcz_store.cc:276] 0505 07:54:11.188111 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.190677 (+ 2566us) service_pool.cc:224] Handling call
0505 07:54:17.895301 (+6704624us) ts_tablet_manager.cc:1151] Deleting tablet ae829daad64545e687a9528e5c38b9ed
0505 07:54:17.895312 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.895407 (+ 95us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6704437}
I20260505 07:54:17.895800 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 251) took 6704 ms. Trace:
I20260505 07:54:17.895869 2268 rpcz_store.cc:276] 0505 07:54:11.191520 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.191569 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:17.895703 (+6704134us) ts_tablet_manager.cc:1151] Deleting tablet a32eeaca77c1475f879465cb1046543b
0505 07:54:17.895712 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.895787 (+ 75us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6703902}
I20260505 07:54:17.896134 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 253) took 6682 ms. Trace:
I20260505 07:54:17.896207 2268 rpcz_store.cc:276] 0505 07:54:11.214062 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.214153 (+ 91us) service_pool.cc:224] Handling call
0505 07:54:17.896063 (+6681910us) ts_tablet_manager.cc:1151] Deleting tablet e278b86e7b5d444ebad8d0f6921d4c62
0505 07:54:17.896070 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.896121 (+ 51us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6681679}
I20260505 07:54:17.896430 2268 tablet_replica.cc:333] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.896618 2268 raft_consensus.cc:2243] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.896828 2268 raft_consensus.cc:2272] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:17.811656 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 135934us
I20260505 07:54:17.842052 1878 raft_consensus.cc:493] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.897445 1878 raft_consensus.cc:515] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.897972 1878 leader_election.cc:290] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:17.862111 990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260505 07:54:17.898232 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_7a2d015cb9324e77afad1280b0e8081c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:17.898473 2268 ts_tablet_manager.cc:1916] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.814700 1181 tablet_bootstrap.cc:492] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:17.898689 1181 ts_tablet_manager.cc:1403] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.477s user 0.059s sys 0.133s
W20260505 07:54:17.898739 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7a2d015cb9324e77afad1280b0e8081c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.899400 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_3cbf7ce6c7c145a39364c848ab951b5f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:17.899829 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3cbf7ce6c7c145a39364c848ab951b5f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:17.864698 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 221630us
I20260505 07:54:17.900801 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66f069982138460da3b7ae7a0b4e3833" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:17.901392 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b48208e68247e78c12f0fec7ba07dc" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:17.901379 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad5c570d4861460d8ab0c8ff8a7a3a03" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:17.899505 1181 raft_consensus.cc:359] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.902213 1181 raft_consensus.cc:385] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.902282 1181 raft_consensus.cc:740] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.902472 1181 consensus_queue.cc:260] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:17.902516 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 68909.3us
I20260505 07:54:17.902614 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f4854e8752493fb832ae2c12a033ce" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:17.902948 1181 ts_tablet_manager.cc:1434] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260505 07:54:17.903201 1181 tablet_bootstrap.cc:492] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:17.903173 3100 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:47472 (request call id 358) took 1753 ms. Trace:
I20260505 07:54:17.903497 3100 rpcz_store.cc:276] 0505 07:54:16.149415 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.149478 (+ 63us) service_pool.cc:224] Handling call
0505 07:54:17.903161 (+1753683us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:16.149924 (+ 0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0505 07:54:16.150049 (+ 125us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0505 07:54:16.150061 (+ 12us) alter_schema_op.cc:66] Acquired schema lock
0505 07:54:16.150070 (+ 9us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0505 07:54:16.150111 (+ 41us) alter_schema_op.cc:128] START. Timestamp: 1777967656150070
0505 07:54:16.150115 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:16.153693 (+ 3578us) log.cc:844] Serialized 173 byte log entry
0505 07:54:17.651386 (+1497693us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.657530 (+ 6144us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0505 07:54:17.658065 (+ 535us) log.cc:844] Serialized 5 byte log entry
0505 07:54:17.898474 (+240409us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.898705 (+ 231us) log.cc:844] Serialized 14 byte log entry
0505 07:54:17.903125 (+ 4420us) alter_schema_op.cc:73] Released schema lock
0505 07:54:17.903130 (+ 5us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6064,"mutex_wait_us":8756,"prepare.queue_time_us":118,"prepare.run_cpu_time_us":83833,"prepare.run_wall_time_us":476896,"raft.queue_time_us":8821,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":200,"replication_time_us":1501233,"thread_start_us":550713,"threads_started":2,"wal-append.queue_time_us":700350,"wal-append.run_cpu_time_us":559,"wal-append.run_wall_time_us":1021971}]]}
I20260505 07:54:17.904664 1181 tablet_bootstrap.cc:654] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.908231 2831 consensus_queue.cc:1048] T 4ef22101fe014e21adea6e10f4b550e7 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.909201 3176 raft_consensus.cc:2804] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:17.909273 3176 raft_consensus.cc:493] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:17.909341 3176 raft_consensus.cc:3060] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:17.920473 3149 raft_consensus.cc:493] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.920603 3149 raft_consensus.cc:515] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.921111 3149 leader_election.cc:290] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:17.930328 3004 consensus_queue.cc:237] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.931464 2872 leader_election.cc:290] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:17.931982 1176 tablet_bootstrap.cc:492] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:17.931990 1878 consensus_queue.cc:1048] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.936717 2268 ts_tablet_manager.cc:1929] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.936992 1172 tablet_bootstrap.cc:492] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:17.937153 1172 ts_tablet_manager.cc:1403] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.061s user 0.002s sys 0.007s
I20260505 07:54:17.937321 1878 consensus_queue.cc:1048] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.938033 1172 raft_consensus.cc:359] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.938227 1172 raft_consensus.cc:385] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.938279 1172 raft_consensus.cc:740] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.937719 991 leader_election.cc:304] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:17.938555 3202 raft_consensus.cc:2804] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:17.938704 3202 raft_consensus.cc:493] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:17.938704 2268 log.cc:1199] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/e6a5f823db6143319d743d363a77ea47
I20260505 07:54:17.938889 3202 raft_consensus.cc:3060] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:17.939153 2268 ts_tablet_manager.cc:1950] T e6a5f823db6143319d743d363a77ea47 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:17.939915 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 254) took 6725 ms. Trace:
I20260505 07:54:17.940367 3176 raft_consensus.cc:515] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.940852 3176 leader_election.cc:290] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:17.941566 3202 raft_consensus.cc:515] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.939998 2268 rpcz_store.cc:276] 0505 07:54:11.214679 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.214720 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:17.896389 (+6681669us) ts_tablet_manager.cc:1151] Deleting tablet e6a5f823db6143319d743d363a77ea47
0505 07:54:17.896398 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.912863 (+ 16465us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.936671 (+ 23808us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.939896 (+ 3225us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6681468}
I20260505 07:54:17.942054 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 255) took 6727 ms. Trace:
I20260505 07:54:17.942138 2268 rpcz_store.cc:276] 0505 07:54:11.214893 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.214937 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:17.941934 (+6726997us) ts_tablet_manager.cc:1151] Deleting tablet 25072055f5534078adf5b5025614f951
0505 07:54:17.941945 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.942037 (+ 92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75,"tablet-delete.queue_time_us":6726819}
I20260505 07:54:17.942286 3202 leader_election.cc:290] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:17.942384 2268 tablet_replica.cc:333] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:17.942572 2268 raft_consensus.cc:2243] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.942714 2268 raft_consensus.cc:2272] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.938586 991 leader_election.cc:304] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:17.945816 1878 raft_consensus.cc:2804] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:17.945899 1878 raft_consensus.cc:493] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:17.945959 1878 raft_consensus.cc:3060] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:17.946471 1176 tablet_bootstrap.cc:654] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:17.948104 991 consensus_peers.cc:597] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 080b2984fcf2478a8def6540da63554d. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:17.948455 1878 raft_consensus.cc:515] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.949411 1878 leader_election.cc:290] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:17.950397 2447 raft_consensus.cc:2243] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:17.950557 2447 raft_consensus.cc:2272] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.951063 2447 ts_tablet_manager.cc:1916] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:17.951438 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_b89ce585388145c6932a0c87ba01a5ff"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:17.951866 1172 consensus_queue.cc:260] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:54:17.952026 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b89ce585388145c6932a0c87ba01a5ff in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.952512 1172 ts_tablet_manager.cc:1434] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.015s user 0.001s sys 0.001s
I20260505 07:54:17.952843 1172 tablet_bootstrap.cc:492] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:17.953474 1349 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6c975d29ebde41ecb90fdaf8c72b2997" } modify_external_catalogs: true
I20260505 07:54:17.953620 1349 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6c975d29ebde41ecb90fdaf8c72b2997" } modify_external_catalogs: true
I20260505 07:54:17.954501 2778 consensus_queue.cc:1048] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:17.954650 3112 tablet.cc:1724] T 76939b7978ee4f7ea0ad29312de2a31a P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260505 07:54:17.956020 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 17327cee69a24e0cb493ff55f948bf6c (table default.Table_0e71e56d0528481eb81fbca5fe427f53 [id=53b44968c1ef49638d9f0e79ac9878ce]) successfully deleted
W20260505 07:54:17.956612 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet bde59d52fd234343b1ca13a7d4aae6bf because the tablet was not found. No further retry: Not found: Tablet not found: bde59d52fd234343b1ca13a7d4aae6bf
I20260505 07:54:17.957334 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 12501fdf2cc5491cb1a06db490e70810 (table default.Table_ac5741221f7e4f54bffba1c88d5cf5c1 [id=bd7fd8e15cb94e018989774fdda0aba3]) successfully deleted
I20260505 07:54:17.957521 2447 ts_tablet_manager.cc:1929] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:17.957605 2447 log.cc:1199] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/f121fdd001ec4ea38ccd24a6afd9e51c
I20260505 07:54:17.957947 2447 ts_tablet_manager.cc:1950] T f121fdd001ec4ea38ccd24a6afd9e51c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:17.958199 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet a7d8265c05b74b28959d0de41fad1fde (table default.Table_4e8ee4d80a174660827b921c9f211ac4 [id=ce8bd78012db468e8a358f51c308a86e]) successfully deleted
I20260505 07:54:17.958628 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 180) took 8405 ms. Trace:
I20260505 07:54:17.958701 2447 rpcz_store.cc:276] 0505 07:54:09.553265 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.553312 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:17.567290 (+8013978us) ts_tablet_manager.cc:1151] Deleting tablet f121fdd001ec4ea38ccd24a6afd9e51c
0505 07:54:17.567299 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.954460 (+387161us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.957481 (+ 3021us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:17.958609 (+ 1128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":382875,"tablet-delete.queue_time_us":8008157}
I20260505 07:54:17.958837 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 067c32ecccb44bca8bf9cc3f6c229326 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]) successfully deleted
I20260505 07:54:17.959092 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 179) took 8406 ms. Trace:
I20260505 07:54:17.959160 2447 rpcz_store.cc:276] 0505 07:54:09.553016 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.553062 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:17.958981 (+8405919us) ts_tablet_manager.cc:1151] Deleting tablet 8eb7eaff8837485a8ae3ccb44c60471d
0505 07:54:17.958991 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.959078 (+ 87us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8399800}
I20260505 07:54:17.959359 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 0db162dfad2448e8ac428b3d3fbe17f7 (table default.Table_73ba90aede024514abb230661a44a4ae [id=03a241143fae44a19531301524ef4adb]) successfully deleted
I20260505 07:54:17.959414 2447 tablet_replica.cc:333] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
W20260505 07:54:17.959910 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8b57c1c618484d5d86a846fb982a2240 because the tablet was not found. No further retry: Not found: Tablet not found: 8b57c1c618484d5d86a846fb982a2240
I20260505 07:54:17.960347 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 62d4bc3154704ba2bfde83bca10f812d (table default.Table_82a545edb422426ab7eaeb4ae68a1a3f [id=5a1cf8871df74960bb29f3288ea4826c]) successfully deleted
I20260505 07:54:17.960950 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 141665fcd59048bf926acd0287f1853e (table default.Table_0ec687e90aea4825b28b36eda22678fe [id=c551b1aa1f2e4eaab4ce7c6727c05465]) successfully deleted
I20260505 07:54:17.961367 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet feeafd614b3e4281a8e45b950e699b63 (table default.Table_ad3e5b8653224a0c929e29e9daf47ff5 [id=99a81018fe1b45638089d579b76029d4]) successfully deleted
I20260505 07:54:17.961891 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet b5a0cbf9c3d74cea92920189da332c1c (table default.Table_e2720de78a7742d5829d7db2ef803f0d [id=f99b727e2aa54591baf6f109c07c2677]) successfully deleted
I20260505 07:54:17.962307 2447 raft_consensus.cc:2243] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:17.962360 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet f92c53c68a4b4e319e98b84128e8e6a4 (table default.Table_1496aa36a9c0478fa72f40d4595c2398 [id=73a1cc56dd244ba389866c21655a3a42]) successfully deleted
I20260505 07:54:17.962703 2447 raft_consensus.cc:2272] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:17.962950 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet bc9ae6404dd64bd8bf4898768e94e4f1 (table default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]) successfully deleted
I20260505 07:54:17.963327 2447 ts_tablet_manager.cc:1916] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:17.964004 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 0baf83b155494fddb3c25144608706f7 because the tablet was not found. No further retry: Not found: Tablet not found: 0baf83b155494fddb3c25144608706f7
W20260505 07:54:17.964495 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet d8c9eaaef250461ca7a37d0cf42b07ee because the tablet was not found. No further retry: Not found: Tablet not found: d8c9eaaef250461ca7a37d0cf42b07ee
I20260505 07:54:17.964860 1358 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_21368b74b6e04fb4a13b91d767f1c592"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:17.964983 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 6fe19306f7df49fdb771d21f6560e697 (table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]) successfully deleted
W20260505 07:54:17.965445 1358 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21368b74b6e04fb4a13b91d767f1c592 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:17.965454 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 5683d7366de84653ace56f0edcd3818c (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]) successfully deleted
I20260505 07:54:17.966445 1172 tablet_bootstrap.cc:654] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.967767 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet b256d072933b4566b5587224274deb66 (table default.Table_6cb216f92da74a78acd72a6132b94c1a [id=c0306ceddf024a1589f83f390b38c602]) successfully deleted
I20260505 07:54:17.968834 3149 raft_consensus.cc:493] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:17.968956 3149 raft_consensus.cc:515] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:17.969509 3149 leader_election.cc:290] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:17.970786 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82e9ae417784b7e83f4843b129d0580" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.970995 913 raft_consensus.cc:2468] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:17.973209 892 tablet_service.cc:1511] Processing CreateTablet for tablet 39eeecbfd10e4df4a89c8598aefcfd88 (DEFAULT_TABLE table=default.Table_7b5084730d534072a607d530beced51c [id=e57e07bed89e4eb7a54d84479272e119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:17.973382 887 tablet_service.cc:1511] Processing CreateTablet for tablet 26ffe4cc177d4b17948666ec38c50ebc (DEFAULT_TABLE table=default.Table_9e7869c6af694bebae3920cd0afa7bbc [id=7c3d66a8774848bbb6f1e621aba59b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:17.973567 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39eeecbfd10e4df4a89c8598aefcfd88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.973209 885 tablet_service.cc:1511] Processing CreateTablet for tablet 1a9c2a1e3a9f47af9a7500130b3d4427 (DEFAULT_TABLE table=default.Table_1d013a93011845449eeb37dd0b8a25b3 [id=c7b53e5d82d04daf8a8e21b65391da48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:17.973989 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a9c2a1e3a9f47af9a7500130b3d4427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.975950 899 tablet_service.cc:1511] Processing CreateTablet for tablet 86927ba8a8364f30b07aa393669c1498 (DEFAULT_TABLE table=default.Table_45058c18f11e47608bcaad243991f75b [id=c92756f25f804794b12e48de32d78a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:17.976050 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a890d5e3f1af49b5aaa28c6c51cb3874" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.976238 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86927ba8a8364f30b07aa393669c1498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.976213 913 raft_consensus.cc:2468] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:17.978008 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26ffe4cc177d4b17948666ec38c50ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.979522 913 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a98243da14d43bc649bb19aed4f0c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:17.979677 3194 tablet.cc:1724] T f84f464e83cb412eade9978ea99f6685 P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:17.979738 913 raft_consensus.cc:2468] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:17.986240 1176 tablet_bootstrap.cc:492] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:17.986860 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet df3a49b59ed6485f970074391f932482 (table default.Table_7af623f581af498b86c8c3b1adbc6b80 [id=306149ae47a745edabf863d5004d5012]) successfully deleted
I20260505 07:54:17.987095 1176 ts_tablet_manager.cc:1403] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.055s user 0.003s sys 0.001s
W20260505 07:54:17.987115 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 91d57f5bb6e847e6b099134d1382b730 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91d57f5bb6e847e6b099134d1382b730 already in progress: creating tablet
I20260505 07:54:17.987627 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]) successfully deleted
W20260505 07:54:17.987659 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 86516.2us
I20260505 07:54:17.988189 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99850198f4d04607b52953342daa753e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:17.988727 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180157193a184add848ba67ca8042109" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:17.986424 718 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 83692.1us
W20260505 07:54:17.989214 854 leader_election.cc:343] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:17.989380 988 leader_election.cc:343] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:17.989445 718 consensus_peers.cc:597] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6fe19306f7df49fdb771d21f6560e697. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:17.989476 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 880e326cbea74233a0ec5abb31f5b0d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 880e326cbea74233a0ec5abb31f5b0d7 already in progress: creating tablet
I20260505 07:54:17.989665 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet abe5f6f0774545b296bf1b03a08e145d (table default.Table_c22dca26148c48f882eeea850f50bea4 [id=6c12928213864a1195f14e35d819f339]) successfully deleted
W20260505 07:54:17.989672 854 leader_election.cc:343] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:17.990031 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f4854e8752493fb832ae2c12a033ce" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:17.991134 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:17.991694 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7887c3462f74740b314e3bec36e280d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:17.991880 912 raft_consensus.cc:3060] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:17.994385 912 raft_consensus.cc:2468] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:17.994454 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 145380us
I20260505 07:54:17.994841 756 tablet_service.cc:1558] Processing DeleteTablet for tablet f1058fcb9fa6457a85489049ca1d9ada with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.995956 1176 raft_consensus.cc:359] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:17.998152 1176 raft_consensus.cc:385] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:17.999842 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 2d86b8b4dadf4fd2b6409a8cb70e313e (table default.Table_ff8ae2444af4400eb86af8cc0fa3c146 [id=6a242917efbf4ae5a5fa82539d6f7b34]) successfully deleted
I20260505 07:54:18.000114 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet d7e0c8ad716e443588d4ae3aa28ed4bf (table default.Table_314825f5a7f644dba828c34f3d2d740a [id=056371ab05fc41c78f31a8ac6887ecb5]) successfully deleted
I20260505 07:54:18.000336 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet ba616702dc6040bb805bb3ffc8a8a871 (table default.Table_2717154d3561495692818d7755d53c41 [id=23c6264859f64b598af609ed2fff6e8f]) successfully deleted
W20260505 07:54:18.000559 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet accb2dedc30e494ebd22c33594cee661 because the tablet was not found. No further retry: Not found: Tablet not found: accb2dedc30e494ebd22c33594cee661
I20260505 07:54:18.000765 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 54e4f7f895d943639653c8c32724baf2 (table default.Table_ceaf4d54d57542928716e3056eac2bde [id=54621fbfaf6a4b4a874207bd986b77be]) successfully deleted
W20260505 07:54:18.001009 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet ae829daad64545e687a9528e5c38b9ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae829daad64545e687a9528e5c38b9ed already in progress: creating tablet
W20260505 07:54:18.001219 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet a32eeaca77c1475f879465cb1046543b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a32eeaca77c1475f879465cb1046543b already in progress: creating tablet
W20260505 07:54:18.001423 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e278b86e7b5d444ebad8d0f6921d4c62 because the tablet was not found. No further retry: Not found: Tablet not found: e278b86e7b5d444ebad8d0f6921d4c62
I20260505 07:54:18.001521 1181 tablet_bootstrap.cc:492] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:18.001698 1181 ts_tablet_manager.cc:1403] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.099s user 0.004s sys 0.013s
I20260505 07:54:18.001698 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet e6a5f823db6143319d743d363a77ea47 (table default.Table_61a6fc82edd5408f991f40a8d15394a5 [id=9930805fdf364f25aea11f859b50bc71]) successfully deleted
W20260505 07:54:18.001991 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 25072055f5534078adf5b5025614f951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25072055f5534078adf5b5025614f951 already in progress: creating tablet
I20260505 07:54:17.996562 744 tablet_service.cc:1558] Processing DeleteTablet for tablet 09b46ff0101a4e529077b9e30492ace7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.003654 1176 raft_consensus.cc:740] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:17.997269 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 9a06f77255544d7dabb71a4f70ada809 with delete_type TABLET_DATA_DELETED (replaced by tablet 12841efcef6041d89fc6d7d9b90448c0) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:17.997769 745 tablet_service.cc:1511] Processing CreateTablet for tablet 1a9c2a1e3a9f47af9a7500130b3d4427 (DEFAULT_TABLE table=default.Table_1d013a93011845449eeb37dd0b8a25b3 [id=c7b53e5d82d04daf8a8e21b65391da48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.003899 1176 consensus_queue.cc:260] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.004145 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a9c2a1e3a9f47af9a7500130b3d4427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.005267 2447 ts_tablet_manager.cc:1929] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.005362 2447 log.cc:1199] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/06904a54b1c5463da31af0bc00fb606f
I20260505 07:54:18.005772 2447 ts_tablet_manager.cc:1950] T 06904a54b1c5463da31af0bc00fb606f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:18.006409 887 tablet_service.cc:1511] Processing CreateTablet for tablet a76add0c46824a969fedf91bc597a75d (DEFAULT_TABLE table=default.Table_ba25fa8c94264de0bab2e95e8d9571ec [id=d97f1bbee59e4b0b9ee8fb95b63636f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.006719 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a76add0c46824a969fedf91bc597a75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.007164 1176 ts_tablet_manager.cc:1434] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.020s user 0.002s sys 0.000s
I20260505 07:54:18.007400 1176 tablet_bootstrap.cc:492] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:17.998291 749 tablet_service.cc:1511] Processing CreateTablet for tablet 26ffe4cc177d4b17948666ec38c50ebc (DEFAULT_TABLE table=default.Table_9e7869c6af694bebae3920cd0afa7bbc [id=7c3d66a8774848bbb6f1e621aba59b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.007850 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26ffe4cc177d4b17948666ec38c50ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.008219 1181 raft_consensus.cc:359] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.008364 1181 raft_consensus.cc:385] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.008409 1181 raft_consensus.cc:740] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.008581 1181 consensus_queue.cc:260] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.009070 1181 ts_tablet_manager.cc:1434] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:54:18.009336 1181 tablet_bootstrap.cc:492] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:18.009443 885 tablet_service.cc:1511] Processing CreateTablet for tablet 48890c946d3542948f8c53a552372867 (DEFAULT_TABLE table=default.Table_6125fe0ed26848bfbb9ed9d193944fa1 [id=d8c37876cf824007b39ebaef7f46f2ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.009732 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48890c946d3542948f8c53a552372867. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.011281 1878 raft_consensus.cc:493] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.011400 1878 raft_consensus.cc:515] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.011905 1878 leader_election.cc:290] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.012444 1878 raft_consensus.cc:493] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.012555 1878 raft_consensus.cc:515] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.012988 1878 leader_election.cc:290] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.013377 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 152176572c904b8296073afad0585817 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:16 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.014195 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 8cf3c334ebb24956b50dbefc23d4d60b with delete_type TABLET_DATA_DELETED (replaced by tablet 260a8d41de464e429491ada4bfe2c761) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.014503 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 9c4fc18f6faa488882d9bffb30793cdd with delete_type TABLET_DATA_DELETED (replaced by tablet 57b5378ec4ae40448311d54c79ea33b0) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.014816 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 83e62b7af9b847b7a9005e4345905c0d with delete_type TABLET_DATA_DELETED (replaced by tablet a54d94d8a888483681858398f667a370) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.015306 1026 tablet_service.cc:1511] Processing CreateTablet for tablet b53db8fdadad4ec09d05f65c84100ff0 (DEFAULT_TABLE table=default.Table_d17cd5686eb94c3b8993d4310f0d31f7 [id=8c0e89be595c4eb2b4d4103121cdccf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.015699 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b53db8fdadad4ec09d05f65c84100ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.019603 892 tablet_service.cc:1511] Processing CreateTablet for tablet 794256584111488aa433596dc43db10c (DEFAULT_TABLE table=default.Table_3783f0e113d542d9ad9b5c85f83624bf [id=476b0959937c4366835e85d54d7bcd56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.020016 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 794256584111488aa433596dc43db10c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:18.021971 988 leader_election.cc:343] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.022370 3173 consensus_queue.cc:1048] T c3be796a788e4465a36bb5be44918be1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.024390 1350 master_service.cc:954] ReplaceTablet: received request to replace tablet 01431ad112e8445487823efe8feb0dbe from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:18.025701 1026 tablet_service.cc:1511] Processing CreateTablet for tablet 27ab0ebc13f349f38cf86b9643700ae1 (DEFAULT_TABLE table=default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.026082 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27ab0ebc13f349f38cf86b9643700ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.032208 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 99850198f4d04607b52953342daa753e with delete_type TABLET_DATA_DELETED (replaced by tablet d73fd67f4b91479c8a53792f4f3779c2) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.032747 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet 29a82ed2b87c487aac84194f2dcb0d6b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.033138 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet dd3fc8a9d9f04636b1f31bb88411c1da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.033974 3149 raft_consensus.cc:493] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.034112 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 1a9c2a1e3a9f47af9a7500130b3d4427 (DEFAULT_TABLE table=default.Table_1d013a93011845449eeb37dd0b8a25b3 [id=c7b53e5d82d04daf8a8e21b65391da48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.034107 3149 raft_consensus.cc:515] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.034410 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a9c2a1e3a9f47af9a7500130b3d4427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.034610 3149 leader_election.cc:290] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:18.035905 1358 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_947d0109c8c343418574a18377c6cccd" } new_table_name: "default.Table_1c26ba8978f749a38cfdb26ff048283c" modify_external_catalogs: true
I20260505 07:54:18.036118 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd2e2514c72419390c1bb9bd13f1fb3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:18.036265 911 raft_consensus.cc:3060] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.038575 911 raft_consensus.cc:2468] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:18.039629 911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a890d5e3f1af49b5aaa28c6c51cb3874" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:18.039790 911 raft_consensus.cc:3060] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.042199 911 raft_consensus.cc:2468] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:18.043399 896 tablet_service.cc:1511] Processing CreateTablet for tablet 9fb980e044a34e969223e1ace953d1c1 (DEFAULT_TABLE table=default.Table_246d9d987a3347cdb0f5b4e0f39bcbaf [id=24dc736f110e424cb1beb5c105324cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.043818 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fb980e044a34e969223e1ace953d1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.043764 899 tablet_service.cc:1511] Processing CreateTablet for tablet 57b5378ec4ae40448311d54c79ea33b0 (DEFAULT_TABLE table=default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.044104 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57b5378ec4ae40448311d54c79ea33b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.045455 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 26ffe4cc177d4b17948666ec38c50ebc (DEFAULT_TABLE table=default.Table_9e7869c6af694bebae3920cd0afa7bbc [id=7c3d66a8774848bbb6f1e621aba59b24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.045838 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26ffe4cc177d4b17948666ec38c50ebc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.051442 1172 tablet_bootstrap.cc:492] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:18.051354 3173 consensus_queue.cc:1048] T d5083f98e8e14ccf99a19cdf2bc59018 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.051573 1172 ts_tablet_manager.cc:1403] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.099s user 0.001s sys 0.004s
I20260505 07:54:18.052320 1172 raft_consensus.cc:359] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.052533 1172 raft_consensus.cc:385] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.052582 1172 raft_consensus.cc:740] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.008783 1176 tablet_bootstrap.cc:654] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:17.998755 748 tablet_service.cc:1511] Processing CreateTablet for tablet 39eeecbfd10e4df4a89c8598aefcfd88 (DEFAULT_TABLE table=default.Table_7b5084730d534072a607d530beced51c [id=e57e07bed89e4eb7a54d84479272e119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.053061 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39eeecbfd10e4df4a89c8598aefcfd88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.054092 3173 consensus_queue.cc:1048] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.054580 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 01431ad112e8445487823efe8feb0dbe
I20260505 07:54:17.999258 750 tablet_service.cc:1511] Processing CreateTablet for tablet 86927ba8a8364f30b07aa393669c1498 (DEFAULT_TABLE table=default.Table_45058c18f11e47608bcaad243991f75b [id=c92756f25f804794b12e48de32d78a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.055553 1350 catalog_manager.cc:6741] ReplaceTablet: tablet 01431ad112e8445487823efe8feb0dbe deleted and replaced by tablet abadd3af175e466db81491b8a69a6d7d
I20260505 07:54:18.055841 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86927ba8a8364f30b07aa393669c1498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.055910 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_8ed3074214464ce3ae998a9580562f57"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:18.056480 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8ed3074214464ce3ae998a9580562f57 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.056473 1019 tablet_service.cc:1511] Processing CreateTablet for tablet 794256584111488aa433596dc43db10c (DEFAULT_TABLE table=default.Table_3783f0e113d542d9ad9b5c85f83624bf [id=476b0959937c4366835e85d54d7bcd56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.056773 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 794256584111488aa433596dc43db10c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.059147 1176 tablet_bootstrap.cc:492] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.059274 1176 ts_tablet_manager.cc:1403] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.052s user 0.000s sys 0.002s
I20260505 07:54:18.059996 1176 raft_consensus.cc:359] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.060099 1023 tablet_service.cc:1511] Processing CreateTablet for tablet 86927ba8a8364f30b07aa393669c1498 (DEFAULT_TABLE table=default.Table_45058c18f11e47608bcaad243991f75b [id=c92756f25f804794b12e48de32d78a9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.060391 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86927ba8a8364f30b07aa393669c1498. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.060431 1172 consensus_queue.cc:260] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.061307 1172 ts_tablet_manager.cc:1434] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.010s user 0.001s sys 0.000s
I20260505 07:54:18.061547 1172 tablet_bootstrap.cc:492] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:18.063201 1172 tablet_bootstrap.cc:654] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.063170 1019 tablet_service.cc:1511] Processing CreateTablet for tablet a76add0c46824a969fedf91bc597a75d (DEFAULT_TABLE table=default.Table_ba25fa8c94264de0bab2e95e8d9571ec [id=d97f1bbee59e4b0b9ee8fb95b63636f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.063329 1176 raft_consensus.cc:385] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.063395 1176 raft_consensus.cc:740] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.063462 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a76add0c46824a969fedf91bc597a75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:17.996236 756 tablet_service.cc:1558] Processing DeleteTablet for tablet b12d2d2b866348d781209fa787da7d7e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.063562 1176 consensus_queue.cc:260] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.044431 2831 raft_consensus.cc:493] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:18.063843 2831 raft_consensus.cc:515] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.064071 1176 ts_tablet_manager.cc:1434] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:54:18.064342 1176 tablet_bootstrap.cc:492] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.064404 2831 leader_election.cc:290] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.065032 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_153cefb63d3a4dfa894369fe70347fed"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:18.065536 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_153cefb63d3a4dfa894369fe70347fed in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.065624 1176 tablet_bootstrap.cc:654] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.069650 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 57b5378ec4ae40448311d54c79ea33b0 (DEFAULT_TABLE table=default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.069964 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57b5378ec4ae40448311d54c79ea33b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.074357 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 01431ad112e8445487823efe8feb0dbe with delete_type TABLET_DATA_DELETED (replaced by tablet abadd3af175e466db81491b8a69a6d7d) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:18.044981 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 182) took 8490 ms. Trace:
I20260505 07:54:18.045864 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a98243da14d43bc649bb19aed4f0c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:18.048965 718 consensus_peers.cc:597] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5683d7366de84653ace56f0edcd3818c. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:18.074671 778 raft_consensus.cc:2468] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:18.074755 718 leader_election.cc:343] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 02412333633a4cdebe95a144fa6c363c
I20260505 07:54:18.074868 718 leader_election.cc:304] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:18.075076 2831 raft_consensus.cc:2749] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:18.075434 718 consensus_peers.cc:597] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d3d11bd19f074bbab101448b45426b7a. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:18.075414 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_eebd7fed9d9d4b4c923a82aa7d302199"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:18.074994 2447 rpcz_store.cc:276] 0505 07:54:09.554246 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.554292 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:17.959372 (+8405080us) ts_tablet_manager.cc:1151] Deleting tablet 06904a54b1c5463da31af0bc00fb606f
0505 07:54:17.959382 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:17.967048 (+ 7666us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.005210 (+ 38162us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.044961 (+ 39751us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38388,"tablet-delete.queue_time_us":8400177}
I20260505 07:54:18.075681 718 leader_election.cc:304] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.075872 2831 raft_consensus.cc:2804] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.075908 718 leader_election.cc:304] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
W20260505 07:54:18.075942 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_eebd7fed9d9d4b4c923a82aa7d302199 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.075982 2831 raft_consensus.cc:697] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:18.076191 2831 consensus_queue.cc:237] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.076717 3133 raft_consensus.cc:2804] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.076864 3133 raft_consensus.cc:697] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:18.077077 3133 consensus_queue.cc:237] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.077292 3173 consensus_queue.cc:1048] T 4b528d6c36ac4920b6ea0a6e26cca269 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:18.078347 988 leader_election.cc:343] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.080387 3173 consensus_queue.cc:1048] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.081223 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 9fb980e044a34e969223e1ace953d1c1 (DEFAULT_TABLE table=default.Table_246d9d987a3347cdb0f5b4e0f39bcbaf [id=24dc736f110e424cb1beb5c105324cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.081629 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fb980e044a34e969223e1ace953d1c1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:18.081857 718 leader_election.cc:343] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: df1b62a30b2842b897a04c793bb5e7ce
I20260505 07:54:18.077064 2447 tablet_replica.cc:333] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:18.082641 2447 raft_consensus.cc:2243] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:18.083336 2447 raft_consensus.cc:2272] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.084362 2447 ts_tablet_manager.cc:1916] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.084779 896 tablet_service.cc:1511] Processing CreateTablet for tablet c4a0ef596e66462a8927773dc86b2546 (DEFAULT_TABLE table=default.Table_48bc29e8382d4b5ba99601d826206b08 [id=6c69a2d52d494a7f8388fd540ca2151e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.085193 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4a0ef596e66462a8927773dc86b2546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.088548 1181 tablet_bootstrap.cc:654] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.090096 1025 tablet_service.cc:1511] Processing CreateTablet for tablet 08dafc2271dc49cfaf55915ef0338ad4 (DEFAULT_TABLE table=default.Table_76f464c1c3b84d418ffffab788dc5ea5 [id=ad824f9d9d814018b5258e2ef41a0072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.090492 1025 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08dafc2271dc49cfaf55915ef0338ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.094496 1176 tablet_bootstrap.cc:492] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.094614 1176 ts_tablet_manager.cc:1403] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.005s
I20260505 07:54:18.095242 1350 master_service.cc:954] ReplaceTablet: received request to replace tablet 5b7a1da5cbc94df9b3b697251087b59a from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:18.090096 1023 tablet_service.cc:1511] Processing CreateTablet for tablet c4a0ef596e66462a8927773dc86b2546 (DEFAULT_TABLE table=default.Table_48bc29e8382d4b5ba99601d826206b08 [id=6c69a2d52d494a7f8388fd540ca2151e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.096345 1023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4a0ef596e66462a8927773dc86b2546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.100689 1176 raft_consensus.cc:359] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.100999 1176 raft_consensus.cc:385] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.101114 1176 raft_consensus.cc:740] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.101382 1176 consensus_queue.cc:260] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.102079 1176 ts_tablet_manager.cc:1434] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.007s user 0.001s sys 0.001s
I20260505 07:54:18.102213 1019 tablet_service.cc:1511] Processing CreateTablet for tablet 48890c946d3542948f8c53a552372867 (DEFAULT_TABLE table=default.Table_6125fe0ed26848bfbb9ed9d193944fa1 [id=d8c37876cf824007b39ebaef7f46f2ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.102524 1019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48890c946d3542948f8c53a552372867. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.106314 887 tablet_service.cc:1511] Processing CreateTablet for tablet 260a8d41de464e429491ada4bfe2c761 (DEFAULT_TABLE table=default.Table_e137326bc9684662bffdc5c59e122c6a [id=199b3d61d922405b9e2a018c1c8c6080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.106719 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260a8d41de464e429491ada4bfe2c761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.110351 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet 01431ad112e8445487823efe8feb0dbe with delete_type TABLET_DATA_DELETED (replaced by tablet abadd3af175e466db81491b8a69a6d7d) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.110891 1034 tablet_service.cc:1511] Processing CreateTablet for tablet 39eeecbfd10e4df4a89c8598aefcfd88 (DEFAULT_TABLE table=default.Table_7b5084730d534072a607d530beced51c [id=e57e07bed89e4eb7a54d84479272e119]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.111172 1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39eeecbfd10e4df4a89c8598aefcfd88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.109279 3013 raft_consensus.cc:493] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.112488 3013 raft_consensus.cc:515] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.113144 3013 leader_election.cc:290] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.110015 3199 raft_consensus.cc:493] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.113381 3199 raft_consensus.cc:515] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.110136 3209 raft_consensus.cc:493] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
W20260505 07:54:18.114534 718 leader_election.cc:343] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 72da22d790624b07bfca6f6b6301c925
I20260505 07:54:18.114284 3209 raft_consensus.cc:515] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:18.115860 718 leader_election.cc:343] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: e29ba55a846548679cb3b86305d0d6ed
I20260505 07:54:18.115654 3209 leader_election.cc:290] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:18.116437 718 leader_election.cc:343] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: a7fe9c7199cc4039ae877dc3d4aab17e
I20260505 07:54:18.116590 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 5b7a1da5cbc94df9b3b697251087b59a
I20260505 07:54:18.116668 3199 leader_election.cc:290] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.117609 988 leader_election.cc:304] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.117862 3173 raft_consensus.cc:2804] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:18.117923 1176 tablet_bootstrap.cc:492] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.117949 3173 raft_consensus.cc:493] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:18.118006 3173 raft_consensus.cc:3060] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.118150 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b7a1da5cbc94df9b3b697251087b59a with delete_type TABLET_DATA_DELETED (replaced by tablet f6d59c9bfc3b4712907d13ea143adb17) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:18.118752 1350 catalog_manager.cc:6741] ReplaceTablet: tablet 5b7a1da5cbc94df9b3b697251087b59a deleted and replaced by tablet f6d59c9bfc3b4712907d13ea143adb17
I20260505 07:54:18.119633 1350 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_3783f0e113d542d9ad9b5c85f83624bf" } modify_external_catalogs: true
I20260505 07:54:18.119848 1350 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_3783f0e113d542d9ad9b5c85f83624bf" } modify_external_catalogs: true
I20260505 07:54:18.120438 3173 raft_consensus.cc:515] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.121032 3173 leader_election.cc:290] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:18.123287 854 leader_election.cc:343] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.124164 1176 tablet_bootstrap.cc:654] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.126212 2268 ts_tablet_manager.cc:1916] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.126299 1181 tablet_bootstrap.cc:492] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:18.126405 1181 ts_tablet_manager.cc:1403] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.117s user 0.001s sys 0.001s
I20260505 07:54:18.127030 1181 raft_consensus.cc:359] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.127224 1181 raft_consensus.cc:385] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.129549 988 leader_election.cc:304] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.129902 3203 raft_consensus.cc:2804] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:18.130013 3203 raft_consensus.cc:493] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:18.130136 3203 raft_consensus.cc:3060] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.130304 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af8af6d1f5d9429083002991e3470f7b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.130777 899 tablet_service.cc:1511] Processing CreateTablet for tablet a54d94d8a888483681858398f667a370 (DEFAULT_TABLE table=default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:18.131009 854 leader_election.cc:343] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.131201 892 tablet_service.cc:1511] Processing CreateTablet for tablet 08dafc2271dc49cfaf55915ef0338ad4 (DEFAULT_TABLE table=default.Table_76f464c1c3b84d418ffffab788dc5ea5 [id=ad824f9d9d814018b5258e2ef41a0072]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:18.132233 854 consensus_peers.cc:597] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 09b46ff0101a4e529077b9e30492ace7. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:18.132543 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a54d94d8a888483681858398f667a370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.132738 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08dafc2271dc49cfaf55915ef0338ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.132694 3203 raft_consensus.cc:515] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.133404 3203 leader_election.cc:290] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.133901 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bd2e2514c72419390c1bb9bd13f1fb3" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:18.134042 1053 raft_consensus.cc:3060] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.136346 1053 raft_consensus.cc:2468] T 0bd2e2514c72419390c1bb9bd13f1fb3 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:18.136854 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet f121fdd001ec4ea38ccd24a6afd9e51c (table default.Table_3a76cf09eaed4c9a8c267714a164dd66 [id=6be00fd9040d4febaef6b16dec369a3a]) successfully deleted
W20260505 07:54:18.137306 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 8eb7eaff8837485a8ae3ccb44c60471d because the tablet was not found. No further retry: Not found: Tablet not found: 8eb7eaff8837485a8ae3ccb44c60471d
I20260505 07:54:18.138567 1176 tablet_bootstrap.cc:492] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.138936 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a890d5e3f1af49b5aaa28c6c51cb3874" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:18.139159 1053 raft_consensus.cc:2393] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:18.139174 1176 ts_tablet_manager.cc:1403] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.001s
I20260505 07:54:18.140399 3202 consensus_queue.cc:1048] T 2045202714c14aa388d1e8bc18e2063d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:18.144796 856 leader_election.cc:343] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.144929 856 leader_election.cc:304] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.145165 3149 raft_consensus.cc:2749] T 66f069982138460da3b7ae7a0b4e3833 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:18.145856 856 leader_election.cc:343] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.145979 856 leader_election.cc:304] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.146195 3149 raft_consensus.cc:2749] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:18.146919 856 leader_election.cc:343] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.147010 856 leader_election.cc:304] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.147228 3149 raft_consensus.cc:2749] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:18.147500 856 leader_election.cc:343] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.147689 856 leader_election.cc:304] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.147894 3149 raft_consensus.cc:2749] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.149114 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e7887c3462f74740b314e3bec36e280d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:18.149277 772 raft_consensus.cc:3060] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.149915 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 794256584111488aa433596dc43db10c
I20260505 07:54:18.150610 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 794256584111488aa433596dc43db10c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:18.151692 1350 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_40605e9a72634999bc80f464976c5dac" } modify_external_catalogs: true
I20260505 07:54:18.151964 1350 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_40605e9a72634999bc80f464976c5dac" } modify_external_catalogs: true
I20260505 07:54:18.152803 3145 raft_consensus.cc:493] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.127308 1181 raft_consensus.cc:740] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.152962 3145 raft_consensus.cc:515] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.153352 1181 consensus_queue.cc:260] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.153510 3145 leader_election.cc:290] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:18.155328 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6546745e894545ff9c5f91bcc6e1a41e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.155797 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bf49b787ef4de095448751464708a4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.156422 1172 tablet_bootstrap.cc:492] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:18.156540 1172 ts_tablet_manager.cc:1403] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.095s user 0.006s sys 0.004s
I20260505 07:54:18.157115 991 leader_election.cc:304] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.157240 1172 raft_consensus.cc:359] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.157374 1172 raft_consensus.cc:385] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.157421 1172 raft_consensus.cc:740] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.157575 1172 consensus_queue.cc:260] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.157660 3202 raft_consensus.cc:2804] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:18.157737 3202 raft_consensus.cc:493] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:18.157788 3202 raft_consensus.cc:3060] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.158078 1172 ts_tablet_manager.cc:1434] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:18.158964 1172 tablet_bootstrap.cc:492] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:18.160192 3202 raft_consensus.cc:515] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.160769 3202 leader_election.cc:290] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.161597 1181 ts_tablet_manager.cc:1434] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.035s user 0.001s sys 0.001s
I20260505 07:54:18.162359 772 raft_consensus.cc:2468] T e7887c3462f74740b314e3bec36e280d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:18.162609 3112 tablet.cc:1724] T f84f464e83cb412eade9978ea99f6685 P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:18.162868 1181 tablet_bootstrap.cc:492] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:18.164984 1181 tablet_bootstrap.cc:654] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.166167 3199 raft_consensus.cc:493] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.166332 3199 raft_consensus.cc:515] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.166894 3199 leader_election.cc:290] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.170187 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a890d5e3f1af49b5aaa28c6c51cb3874" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:18.171362 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a98243da14d43bc649bb19aed4f0c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:18.171478 777 raft_consensus.cc:3060] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.172238 1176 raft_consensus.cc:359] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.172437 1176 raft_consensus.cc:385] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.172494 1176 raft_consensus.cc:740] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.172683 1176 consensus_queue.cc:260] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.173231 1176 ts_tablet_manager.cc:1434] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.034s user 0.001s sys 0.000s
I20260505 07:54:18.174234 1172 tablet_bootstrap.cc:654] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.174249 777 raft_consensus.cc:2468] T 0e9a98243da14d43bc649bb19aed4f0c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:18.175732 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82e9ae417784b7e83f4843b129d0580" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:18.175905 920 raft_consensus.cc:2468] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:18.178191 718 leader_election.cc:304] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.179687 3133 raft_consensus.cc:2804] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:18.179783 3133 raft_consensus.cc:493] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:18.179862 3133 raft_consensus.cc:3060] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.182443 1176 tablet_bootstrap.cc:492] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.182953 3133 raft_consensus.cc:515] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.184201 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82e9ae417784b7e83f4843b129d0580" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:18.184403 920 raft_consensus.cc:3060] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.187633 920 raft_consensus.cc:2468] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:18.183552 3133 leader_election.cc:290] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.167420 3202 consensus_queue.cc:1048] T 2706bc9b9e02414693062deff5cfe81b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.168398 991 leader_election.cc:304] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:18.169771 1053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99850198f4d04607b52953342daa753e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.170110 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180157193a184add848ba67ca8042109" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.170406 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af8af6d1f5d9429083002991e3470f7b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.184463 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8851ec03694289ae33a86f2faa4b09" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.197322 1172 tablet_bootstrap.cc:492] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:18.197515 1172 ts_tablet_manager.cc:1403] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.039s user 0.000s sys 0.003s
I20260505 07:54:18.197906 3202 raft_consensus.cc:2804] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.198011 3202 raft_consensus.cc:697] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:18.198205 3202 consensus_queue.cc:237] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.198376 1172 raft_consensus.cc:359] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.198558 1172 raft_consensus.cc:385] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.198616 1172 raft_consensus.cc:740] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:54:18.199901 1315 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_65d14038399640c284b8382aa80e344f [id=fc9aba6852ee42cc83697f86e0e69c5e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 68e608ac01ef4a88906e585bd7ee25ac consensus state has no leader
I20260505 07:54:18.200250 420 external_mini_cluster.cc:1366] Running /tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
/tmp/dist-test-tasklSS7ae/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.105.60:35401
--webserver_interface=127.0.105.60
--webserver_port=44877
--builtin_ntp_servers=127.0.105.20:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.105.62:33621,127.0.105.61:33565,127.0.105.60:35401
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260505 07:54:18.200970 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:18.201383 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a (table default.Table_bffe918db391471ba1407a66a5b7c240 [id=a238daba389c412ea67318d99cfb5559]). Expected version 1 got 0
I20260505 07:54:18.201596 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:18.201884 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e243711164f04faabbde53b0a09d90b1 (table default.Table_a3cc6e739a71414f9f925aa759a07bd3 [id=5993b31e5350440f9715f1e416b90eb2]). Expected version 1 got 0
I20260505 07:54:18.202080 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:18.202387 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:18.202509 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:18.202749 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:18.203019 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 66082a7a75e345639e7a268727922fbb (table default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6]). Expected version 3 got 0
I20260505 07:54:18.203195 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f84f464e83cb412eade9978ea99f6685 (table default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891]). Expected version 1 got 0
I20260505 07:54:18.203327 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:18.203434 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:18.203559 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:18.204051 1176 tablet_bootstrap.cc:654] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.204039 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:18.204222 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:18.204350 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 429ccdb56f0b456483cf1debb5de0beb (table default.Table_6235f4345c354a6daa96d2ece798f82e [id=1a0d30f064af4c2f8e082d5742ba6d28]). Expected version 1 got 0
I20260505 07:54:18.204677 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:18.205054 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:18.205375 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:18.205699 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:18.205904 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:18.206182 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e38f248556bd4e49b06da72a1cbd7d8f (table default.Table_5c2b133923064d5eb9eabc21fb62f098 [id=12443115e63e4428a9cb798980fcbf5b]). Expected version 1 got 0
I20260505 07:54:18.206362 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:18.206501 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_65d14038399640c284b8382aa80e344f [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 2 got 0
I20260505 07:54:18.206792 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_90c98f62112f465294c60c56f6835a93 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 2 got 0
I20260505 07:54:18.207022 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:18.207301 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_947d0109c8c343418574a18377c6cccd [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 1 got 0
I20260505 07:54:18.207724 1172 consensus_queue.cc:260] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.208479 1172 ts_tablet_manager.cc:1434] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.011s user 0.001s sys 0.002s
I20260505 07:54:18.208736 1172 tablet_bootstrap.cc:492] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:18.210242 1172 tablet_bootstrap.cc:654] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:18.211452 991 leader_election.cc:343] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: e8bf49b787ef4de095448751464708a4
W20260505 07:54:18.211867 991 leader_election.cc:343] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 6546745e894545ff9c5f91bcc6e1a41e
I20260505 07:54:18.212146 718 leader_election.cc:304] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.212739 777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82e9ae417784b7e83f4843b129d0580" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:18.212947 3145 raft_consensus.cc:493] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.213095 777 raft_consensus.cc:2393] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:18.213075 3145 raft_consensus.cc:515] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.213485 3133 raft_consensus.cc:2804] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.213660 3145 leader_election.cc:290] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:18.213660 3133 raft_consensus.cc:697] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:18.213581 3113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "23c57a2c927c4c54bccecfeb3a775311" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } } }
I20260505 07:54:18.213819 3113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 [sys.catalog]: This master's current role is: LEADER
I20260505 07:54:18.214100 3133 consensus_queue.cc:237] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.216773 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:18.224336 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a890d5e3f1af49b5aaa28c6c51cb3874" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:18.224625 920 raft_consensus.cc:2393] T a890d5e3f1af49b5aaa28c6c51cb3874 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:18.225685 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e9a98243da14d43bc649bb19aed4f0c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:18.225836 920 raft_consensus.cc:3060] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.228497 920 raft_consensus.cc:2468] T 0e9a98243da14d43bc649bb19aed4f0c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:18.236961 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82e9ae417784b7e83f4843b129d0580" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:18.237144 920 raft_consensus.cc:2393] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
W20260505 07:54:18.239185 425 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 368820us
I20260505 07:54:18.243952 781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a85b65b1904bd6b793af48f71ed726" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.244582 767 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b8851ec03694289ae33a86f2faa4b09" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.244879 3199 raft_consensus.cc:493] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:18.245035 3199 raft_consensus.cc:515] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.245616 3199 leader_election.cc:290] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.246158 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:18.249688 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
W20260505 07:54:18.250262 854 leader_election.cc:343] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.252466 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:18.252746 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:18.253393 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 490483562d8f4c2daa7eaa333a9f3a0c (table default.Table_3acf7c3eb13d473a9d952098c7524753 [id=fe7ee6e8118840e8a9f70a007d80c2b3]). Expected version 1 got 0
I20260505 07:54:18.253652 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 90612f0afb4949f491e625c61e40d247 (table default.Table_61a6fc82edd5408f991f40a8d15394a5 [id=9930805fdf364f25aea11f859b50bc71]). Expected version 1 got 0
I20260505 07:54:18.253849 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:18.255280 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:18.255487 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1506cd31fe364e328867bbf70e36845e (table default.Table_eee3bd15017341fca226f671948e94e9 [id=c7cad0401e40485e9ed03160b9bed704]). Expected version 1 got 0
I20260505 07:54:18.255658 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:18.256927 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 76939b7978ee4f7ea0ad29312de2a31a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]). Expected version 2 got 0
I20260505 07:54:18.257223 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
W20260505 07:54:18.257236 718 leader_election.cc:343] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: ba616702dc6040bb805bb3ffc8a8a871
I20260505 07:54:18.257536 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
W20260505 07:54:18.262379 988 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.1:36247, user_credentials={real_user=slave}} blocked reactor thread for 72853.3us
W20260505 07:54:18.263558 988 leader_election.cc:343] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.264250 988 leader_election.cc:304] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.273233 2778 raft_consensus.cc:2749] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.258276 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:18.274622 988 leader_election.cc:304] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.274777 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:18.274912 2778 raft_consensus.cc:2804] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.275005 2778 raft_consensus.cc:697] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:18.274989 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:18.275225 2778 consensus_queue.cc:237] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.275450 2447 ts_tablet_manager.cc:1929] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:18.275544 2447 log.cc:1199] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/ff7ed926a3ff4e7993864de734ca3c50
I20260505 07:54:18.275975 2447 ts_tablet_manager.cc:1950] T ff7ed926a3ff4e7993864de734ca3c50 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:18.275184 1348 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1bd5951bfc3346c2a6aad8f33357394a (table default.Table_4e651b0d817f408e8e36b0399a57a04f [id=d85dae5f1f424012929e0d90ef8b2d22]). Expected version 1 got 0
I20260505 07:54:18.276867 1354 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_6235f4345c354a6daa96d2ece798f82e [id=1a0d30f064af4c2f8e082d5742ba6d28] alter complete (version 1)
W20260505 07:54:18.277832 717 consensus_peers.cc:597] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6fe19306f7df49fdb771d21f6560e697. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:18.279929 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 06904a54b1c5463da31af0bc00fb606f (table default.Table_577ff393d2af447a915f4460ab7692f1 [id=96336409bde3400fb18cb06e0d84e99c]) successfully deleted
I20260505 07:54:18.281730 3203 raft_consensus.cc:493] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.281909 3203 raft_consensus.cc:515] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.282070 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72da22d790624b07bfca6f6b6301c925" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.282485 3203 leader_election.cc:290] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.283149 1031 tablet_service.cc:1511] Processing CreateTablet for tablet 260a8d41de464e429491ada4bfe2c761 (DEFAULT_TABLE table=default.Table_e137326bc9684662bffdc5c59e122c6a [id=199b3d61d922405b9e2a018c1c8c6080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.283500 1349 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet d37dc6df9da5401384a6fc5de868b41c
I20260505 07:54:18.283882 1026 tablet_service.cc:1511] Processing CreateTablet for tablet a54d94d8a888483681858398f667a370 (DEFAULT_TABLE table=default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.283968 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82e9ae417784b7e83f4843b129d0580" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.284138 1049 raft_consensus.cc:2393] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:18.284415 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b7a1da5cbc94df9b3b697251087b59a with delete_type TABLET_DATA_DELETED (replaced by tablet f6d59c9bfc3b4712907d13ea143adb17) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.284400 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_02afe59354994bffb24b190e9a7abf85" } new_table_name: "default.Table_e9f1ff58ca6d41f19d05c458dc529941" modify_external_catalogs: true
I20260505 07:54:18.284880 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e82e9ae417784b7e83f4843b129d0580" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:18.284978 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 794256584111488aa433596dc43db10c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.285055 1049 raft_consensus.cc:2393] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:18.285696 991 leader_election.cc:304] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.286090 920 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e8f9f7b30f4912af916c15f9ae25d8" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:18.286258 920 raft_consensus.cc:2468] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:18.286417 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df1b62a30b2842b897a04c793bb5e7ce" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.286559 1050 raft_consensus.cc:2468] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
W20260505 07:54:18.287552 854 leader_election.cc:343] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.293865 3203 raft_consensus.cc:2749] T a890d5e3f1af49b5aaa28c6c51cb3874 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.295948 1355 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 3766ff57fcd5402d98e005a2966568d5
I20260505 07:54:18.296177 991 leader_election.cc:304] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:18.296372 3173 raft_consensus.cc:2804] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:18.296432 3173 raft_consensus.cc:493] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:18.296489 3173 raft_consensus.cc:3060] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.296523 1355 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 230) took 1044 ms. Trace:
I20260505 07:54:18.296649 1355 rpcz_store.cc:276] 0505 07:54:17.251823 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.739952 (+488129us) service_pool.cc:224] Handling call
0505 07:54:17.852354 (+112402us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:17.852488 (+ 134us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:17.852534 (+ 46us) catalog_manager.cc:2824] Locking tablets
0505 07:54:17.971473 (+118939us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:18.036991 (+ 65518us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:18.037075 (+ 84us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:18.295881 (+258806us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:18.296483 (+ 602us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.981558 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.981644 (+ 86us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.981651 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:17.981653 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.981734 (+ 81us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.981742 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.981746 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.981841 (+ 95us) tablet.cc:701] Row locks acquired
0505 07:54:17.981843 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.981869 (+ 26us) write_op.cc:270] Start()
0505 07:54:17.981891 (+ 22us) write_op.cc:276] Timestamp: P: 1777967657981843 usec, L: 0
0505 07:54:17.981894 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.982028 (+ 134us) log.cc:844] Serialized 725 byte log entry
0505 07:54:18.034705 (+ 52677us) op_driver.cc:464] REPLICATION: finished
0505 07:54:18.035432 (+ 727us) write_op.cc:301] APPLY: starting
0505 07:54:18.035440 (+ 8us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:18.035519 (+ 79us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:18.035538 (+ 19us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:18.035636 (+ 98us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:18.035661 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:18.035667 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:18.035716 (+ 49us) log.cc:844] Serialized 29 byte log entry
0505 07:54:18.035749 (+ 33us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:18.035760 (+ 11us) write_op.cc:454] Released schema lock
0505 07:54:18.035764 (+ 4us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:18.035816 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:18.035819 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":698,"apply.run_cpu_time_us":353,"apply.run_wall_time_us":376,"num_ops":2,"prepare.queue_time_us":9504,"prepare.run_cpu_time_us":545,"prepare.run_wall_time_us":545,"raft.queue_time_us":77,"raft.run_cpu_time_us":2,"raft.run_wall_time_us":2,"replication_time_us":52790}]]}
I20260505 07:54:18.297324 3144 raft_consensus.cc:493] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.297451 3144 raft_consensus.cc:515] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.297606 1355 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_f56ca420b05b4508a41ee72179b9c93f" } new_table_name: "default.Table_3f9462e44d2b4d138eb0510c4a571080" modify_external_catalogs: true
I20260505 07:54:18.298930 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 3766ff57fcd5402d98e005a2966568d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:18.299275 3173 raft_consensus.cc:515] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.300004 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet d37dc6df9da5401384a6fc5de868b41c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.300379 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet 3766ff57fcd5402d98e005a2966568d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.300781 3173 leader_election.cc:290] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.304816 988 leader_election.cc:304] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.305524 988 leader_election.cc:304] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.305804 3203 raft_consensus.cc:2749] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.307735 1352 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_3ad26a52aa5e4b0692fb55aa761970ed [id=2aaa5e01c641425cb50db866ad9281d6] alter complete (version 3)
I20260505 07:54:18.314491 3202 raft_consensus.cc:493] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.314669 3202 raft_consensus.cc:515] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.315135 3202 leader_election.cc:290] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.316895 3203 raft_consensus.cc:2749] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260505 07:54:18.317199 896 tablet_service.cc:1558] Processing DeleteTablet for tablet d37dc6df9da5401384a6fc5de868b41c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:18.317625 1351 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 42360f8ee4ff47c2af8d287f792207de
I20260505 07:54:18.318115 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e8f9f7b30f4912af916c15f9ae25d8" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:18.318234 1351 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 229) took 1066 ms. Trace:
I20260505 07:54:18.318289 907 raft_consensus.cc:3060] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.318356 1351 rpcz_store.cc:276] 0505 07:54:17.251739 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.728836 (+477097us) service_pool.cc:224] Handling call
0505 07:54:17.729290 (+ 454us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:17.729478 (+ 188us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:17.729542 (+ 64us) catalog_manager.cc:2824] Locking tablets
0505 07:54:17.843785 (+114243us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:17.894926 (+ 51141us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:17.895025 (+ 99us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:18.317548 (+422523us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:18.318199 (+ 651us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.868762 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.868846 (+ 84us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.868851 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:17.868853 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.868936 (+ 83us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.868951 (+ 15us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.868955 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:17.869051 (+ 96us) tablet.cc:701] Row locks acquired
0505 07:54:17.869053 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.869078 (+ 25us) write_op.cc:270] Start()
0505 07:54:17.869101 (+ 23us) write_op.cc:276] Timestamp: P: 1777967657869013 usec, L: 0
0505 07:54:17.869104 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.869230 (+ 126us) log.cc:844] Serialized 765 byte log entry
0505 07:54:17.892844 (+ 23614us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.893193 (+ 349us) write_op.cc:301] APPLY: starting
0505 07:54:17.893205 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.893241 (+ 36us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.893267 (+ 26us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.893330 (+ 63us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.893364 (+ 34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:17.893373 (+ 9us) write_op.cc:312] APPLY: finished
0505 07:54:17.893434 (+ 61us) log.cc:844] Serialized 29 byte log entry
0505 07:54:17.893486 (+ 52us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.893499 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:17.893504 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.893560 (+ 56us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.893564 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":304,"apply.run_cpu_time_us":362,"apply.run_wall_time_us":361,"mutex_wait_us":24273,"num_ops":2,"prepare.queue_time_us":24319,"prepare.run_cpu_time_us":545,"prepare.run_wall_time_us":545,"raft.queue_time_us":48,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":23714}]]}
I20260505 07:54:18.319418 1351 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_e13508677ec64468adda12ac9efaeb05" } new_table_name: "default.Table_96e345e18b864cce8826bf32cac7ef52" modify_external_catalogs: true
I20260505 07:54:18.320415 1176 tablet_bootstrap.cc:492] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.320544 1176 ts_tablet_manager.cc:1403] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.138s user 0.007s sys 0.009s
I20260505 07:54:18.321362 1176 raft_consensus.cc:359] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.321547 1176 raft_consensus.cc:385] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.321602 1176 raft_consensus.cc:740] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.321828 1176 consensus_queue.cc:260] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.322376 1176 ts_tablet_manager.cc:1434] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:18.324647 1176 tablet_bootstrap.cc:492] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.324771 1031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260a8d41de464e429491ada4bfe2c761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.325249 1026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a54d94d8a888483681858398f667a370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.326047 1176 tablet_bootstrap.cc:654] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.326779 1050 raft_consensus.cc:1240] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Rejecting Update request from peer eac7e1e8caba43cdb4df80cbb80e891c for earlier term 1. Current term is 2. Ops: []
I20260505 07:54:18.327874 3004 raft_consensus.cc:493] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:18.328016 3004 raft_consensus.cc:515] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:18.328891 3004 leader_election.cc:290] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:18.329275 718 leader_election.cc:343] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 1852f7f4ff594639908b03730b68fe86
I20260505 07:54:18.331985 750 tablet_service.cc:1511] Processing CreateTablet for tablet 794256584111488aa433596dc43db10c (DEFAULT_TABLE table=default.Table_3783f0e113d542d9ad9b5c85f83624bf [id=476b0959937c4366835e85d54d7bcd56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.332340 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 794256584111488aa433596dc43db10c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:18.333202 1351 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 025c94c839484adba26945b62a217f5c consensus state has no leader
I20260505 07:54:18.333559 1351 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_a63821ea5fc747bdb435a2f668326eea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:18.334100 1351 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a63821ea5fc747bdb435a2f668326eea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.336854 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 42360f8ee4ff47c2af8d287f792207de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.337037 3173 raft_consensus.cc:493] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.337085 1176 tablet_bootstrap.cc:492] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.337222 1176 ts_tablet_manager.cc:1403] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260505 07:54:18.337164 3173 raft_consensus.cc:515] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.337821 3173 leader_election.cc:290] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.338779 750 tablet_service.cc:1511] Processing CreateTablet for tablet a76add0c46824a969fedf91bc597a75d (DEFAULT_TABLE table=default.Table_ba25fa8c94264de0bab2e95e8d9571ec [id=d97f1bbee59e4b0b9ee8fb95b63636f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.339103 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a76add0c46824a969fedf91bc597a75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.337869 1176 raft_consensus.cc:359] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.340907 1176 raft_consensus.cc:385] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.340981 1176 raft_consensus.cc:740] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.341022 3145 raft_consensus.cc:493] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.341145 1176 consensus_queue.cc:260] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.341143 3145 raft_consensus.cc:515] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.341611 3145 leader_election.cc:290] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.342957 1181 tablet_bootstrap.cc:492] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:18.342975 1176 ts_tablet_manager.cc:1434] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260505 07:54:18.343119 1181 ts_tablet_manager.cc:1403] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.180s user 0.005s sys 0.001s
I20260505 07:54:18.343789 1181 raft_consensus.cc:359] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.343947 1181 raft_consensus.cc:385] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.344005 1181 raft_consensus.cc:740] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.344190 1181 consensus_queue.cc:260] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.344668 1181 ts_tablet_manager.cc:1434] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260505 07:54:18.345685 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1506cd31fe364e328867bbf70e36845e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:18.346170 854 leader_election.cc:343] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.346472 1176 tablet_bootstrap.cc:492] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.346599 1351 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_145774eb3d744a92b4b09f7a8ee032f5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:18.348066 1176 tablet_bootstrap.cc:654] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:18.348559 1351 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_145774eb3d744a92b4b09f7a8ee032f5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.350885 748 tablet_service.cc:1511] Processing CreateTablet for tablet 57b5378ec4ae40448311d54c79ea33b0 (DEFAULT_TABLE table=default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.351233 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57b5378ec4ae40448311d54c79ea33b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.359840 1351 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_79228e2b3f8b49b68041c5310cc30d88"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:18.360478 1351 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79228e2b3f8b49b68041c5310cc30d88 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.363008 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e8f9f7b30f4912af916c15f9ae25d8" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:18.363195 1358 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet aacf715cb07d4fe9b2c57caea594ab87 consensus state has no leader
I20260505 07:54:18.363317 782 raft_consensus.cc:2468] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:18.363552 1358 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_26167dd500974094b3adfce3f4247587"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:18.363936 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e8f9f7b30f4912af916c15f9ae25d8" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:18.364132 775 raft_consensus.cc:3060] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:18.364113 1358 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_26167dd500974094b3adfce3f4247587 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:18.366031 1353 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 03260a8507804e548486c81225de3ad9 consensus state has no leader
I20260505 07:54:18.366230 1353 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 231) took 1085 ms. Trace:
I20260505 07:54:18.366315 1353 rpcz_store.cc:276] 0505 07:54:17.280721 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.848534 (+567813us) service_pool.cc:224] Handling call
0505 07:54:17.848917 (+ 383us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:17.849138 (+ 221us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:17.849312 (+ 174us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:17.849383 (+ 71us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:17.897345 (+ 47962us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:18.366173 (+468828us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.871335 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.871423 (+ 88us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.871428 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:17.871430 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.871503 (+ 73us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.871511 (+ 8us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.871515 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:17.871555 (+ 40us) tablet.cc:701] Row locks acquired
0505 07:54:17.871558 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:17.871599 (+ 41us) write_op.cc:270] Start()
0505 07:54:17.871621 (+ 22us) write_op.cc:276] Timestamp: P: 1777967657871515 usec, L: 0
0505 07:54:17.871624 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.871743 (+ 119us) log.cc:844] Serialized 405 byte log entry
0505 07:54:17.892973 (+ 21230us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.894593 (+ 1620us) write_op.cc:301] APPLY: starting
0505 07:54:17.894603 (+ 10us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.894627 (+ 24us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.894654 (+ 27us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.894684 (+ 30us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.894707 (+ 23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:17.894715 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:17.894761 (+ 46us) log.cc:844] Serialized 23 byte log entry
0505 07:54:17.894796 (+ 35us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.894806 (+ 10us) write_op.cc:454] Released schema lock
0505 07:54:17.894810 (+ 4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1574,"apply.run_cpu_time_us":283,"apply.run_wall_time_us":283,"mutex_wait_us":21543,"num_ops":1,"prepare.queue_time_us":21581,"prepare.run_cpu_time_us":477,"prepare.run_wall_time_us":477,"raft.queue_time_us":276,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":21337,"thread_start_us":4748,"threads_started":1}]]}
I20260505 07:54:18.367168 775 raft_consensus.cc:2468] T 68e8f9f7b30f4912af916c15f9ae25d8 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:18.367358 1353 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_2b5ac39d20eb4deeb452b514f976ba97"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:18.367853 1353 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2b5ac39d20eb4deeb452b514f976ba97 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.368470 3144 leader_election.cc:290] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:18.369124 856 leader_election.cc:343] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.369235 856 leader_election.cc:304] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.369367 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 42360f8ee4ff47c2af8d287f792207de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:18.370149 1352 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0] alter complete (version 1)
I20260505 07:54:18.370296 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "898ebd1a70984cc1b9a71009b1070bfe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:18.369617 856 leader_election.cc:343] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.370931 856 leader_election.cc:304] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:18.371317 856 leader_election.cc:343] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.371410 856 leader_election.cc:304] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.371845 3147 raft_consensus.cc:2749] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.372020 3147 raft_consensus.cc:2749] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.369742 3145 raft_consensus.cc:2749] T 99850198f4d04607b52953342daa753e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:18.370688 854 leader_election.cc:343] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:18.375154 856 leader_election.cc:343] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.375290 856 leader_election.cc:304] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.375857 3145 raft_consensus.cc:2749] T 7b8851ec03694289ae33a86f2faa4b09 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.377003 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25072055f5534078adf5b5025614f951" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.377697 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 181) took 8823 ms. Trace:
I20260505 07:54:18.377779 2447 rpcz_store.cc:276] 0505 07:54:09.553953 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.554008 (+ 55us) service_pool.cc:224] Handling call
0505 07:54:18.077022 (+8523014us) ts_tablet_manager.cc:1151] Deleting tablet ff7ed926a3ff4e7993864de734ca3c50
0505 07:54:18.077032 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.094067 (+ 17035us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.275393 (+181326us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.377679 (+102286us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8516579}
I20260505 07:54:18.378067 2447 tablet_replica.cc:333] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:18.378228 2447 raft_consensus.cc:2243] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.378346 2447 raft_consensus.cc:2272] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.379374 907 raft_consensus.cc:2468] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:18.379420 2447 ts_tablet_manager.cc:1916] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:18.380599 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 89d3e08134704859abc44257f484b8cd consensus state has no leader
I20260505 07:54:18.380728 748 tablet_service.cc:1511] Processing CreateTablet for tablet 9fb980e044a34e969223e1ace953d1c1 (DEFAULT_TABLE table=default.Table_246d9d987a3347cdb0f5b4e0f39bcbaf [id=24dc736f110e424cb1beb5c105324cbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.380949 1349 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_12419a358167451da817d86794464e07"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:18.381117 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fb980e044a34e969223e1ace953d1c1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:18.381443 1349 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_12419a358167451da817d86794464e07 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.388360 1181 tablet_bootstrap.cc:492] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:18.389981 1181 tablet_bootstrap.cc:654] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:18.357648 991 leader_election.cc:343] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 25072055f5534078adf5b5025614f951
I20260505 07:54:18.390499 756 tablet_service.cc:1558] Processing DeleteTablet for tablet 01431ad112e8445487823efe8feb0dbe with delete_type TABLET_DATA_DELETED (replaced by tablet abadd3af175e466db81491b8a69a6d7d) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.391072 756 tablet_service.cc:1511] Processing CreateTablet for tablet c4a0ef596e66462a8927773dc86b2546 (DEFAULT_TABLE table=default.Table_48bc29e8382d4b5ba99601d826206b08 [id=6c69a2d52d494a7f8388fd540ca2151e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.391810 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet ae829daad64545e687a9528e5c38b9ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.391788 749 tablet_service.cc:1511] Processing CreateTablet for tablet 08dafc2271dc49cfaf55915ef0338ad4 (DEFAULT_TABLE table=default.Table_76f464c1c3b84d418ffffab788dc5ea5 [id=ad824f9d9d814018b5258e2ef41a0072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.391978 1350 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 73042a7ff5fb43f7a781b4f98f4f4020
I20260505 07:54:18.392388 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet abe5f6f0774545b296bf1b03a08e145d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.392685 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 2d86b8b4dadf4fd2b6409a8cb70e313e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.392730 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_969875795fc0470298d7aecc64918154"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:18.393211 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_969875795fc0470298d7aecc64918154 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.393339 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet d7e0c8ad716e443588d4ae3aa28ed4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.393612 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet ba616702dc6040bb805bb3ffc8a8a871 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.394212 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 6fe19306f7df49fdb771d21f6560e697 with delete_type TABLET_DATA_DELETED (replaced by tablet 317144923d724f6fa96ad49809d25b46) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.394878 892 tablet_service.cc:1558] Processing DeleteTablet for tablet 73042a7ff5fb43f7a781b4f98f4f4020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:18.395066 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet df3a49b59ed6485f970074391f932482 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.395174 2276 ts_tablet_manager.cc:1929] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260505 07:54:18.395555 1355 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9926e569032144ec9d3148f4561333ab consensus state has no leader
I20260505 07:54:18.396018 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_e754ed46e4014d0ba9652ccb3db35817"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:18.396433 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e754ed46e4014d0ba9652ccb3db35817 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.397934 2447 ts_tablet_manager.cc:1929] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:18.399811 2268 ts_tablet_manager.cc:1929] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:18.399958 2268 log.cc:1199] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/72da22d790624b07bfca6f6b6301c925
I20260505 07:54:18.400300 2268 ts_tablet_manager.cc:1950] T 72da22d790624b07bfca6f6b6301c925 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:18.400579 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 256) took 7185 ms. Trace:
I20260505 07:54:18.400683 2268 rpcz_store.cc:276] 0505 07:54:11.215111 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.215196 (+ 85us) service_pool.cc:224] Handling call
0505 07:54:17.942345 (+6727149us) ts_tablet_manager.cc:1151] Deleting tablet 72da22d790624b07bfca6f6b6301c925
0505 07:54:17.942354 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.318514 (+376160us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.399756 (+ 81242us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.400566 (+ 810us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6726999}
I20260505 07:54:18.401058 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 257) took 7185 ms. Trace:
I20260505 07:54:18.401141 2268 rpcz_store.cc:276] 0505 07:54:11.215389 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.215504 (+ 115us) service_pool.cc:224] Handling call
0505 07:54:18.400975 (+7185471us) ts_tablet_manager.cc:1151] Deleting tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf
0505 07:54:18.400983 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.401045 (+ 62us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7185261}
I20260505 07:54:18.401440 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 258) took 7185 ms. Trace:
I20260505 07:54:18.401517 2268 rpcz_store.cc:276] 0505 07:54:11.216151 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.216258 (+ 107us) service_pool.cc:224] Handling call
0505 07:54:18.401357 (+7185099us) ts_tablet_manager.cc:1151] Deleting tablet b12d2d2b866348d781209fa787da7d7e
0505 07:54:18.401365 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.401429 (+ 64us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7184910}
I20260505 07:54:18.401746 2268 tablet_replica.cc:333] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:18.401939 2268 raft_consensus.cc:2243] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.410214 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:18.399417 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60772a9697ba417b879dc6025eeb8eec" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.402324 745 tablet_service.cc:1511] Processing CreateTablet for tablet 260a8d41de464e429491ada4bfe2c761 (DEFAULT_TABLE table=default.Table_e137326bc9684662bffdc5c59e122c6a [id=199b3d61d922405b9e2a018c1c8c6080]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.402592 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 5b7a1da5cbc94df9b3b697251087b59a with delete_type TABLET_DATA_DELETED (replaced by tablet f6d59c9bfc3b4712907d13ea143adb17) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.402930 744 tablet_service.cc:1511] Processing CreateTablet for tablet a54d94d8a888483681858398f667a370 (DEFAULT_TABLE table=default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.403194 743 tablet_service.cc:1558] Processing DeleteTablet for tablet 794256584111488aa433596dc43db10c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.403398 761 tablet_service.cc:1558] Processing DeleteTablet for tablet d37dc6df9da5401384a6fc5de868b41c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.403638 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 3766ff57fcd5402d98e005a2966568d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.404124 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 42360f8ee4ff47c2af8d287f792207de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:17 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.416544 2447 log.cc:1199] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/60772a9697ba417b879dc6025eeb8eec
I20260505 07:54:18.417163 2447 ts_tablet_manager.cc:1950] T 60772a9697ba417b879dc6025eeb8eec P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:18.395275 2276 log.cc:1199] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/b256d072933b4566b5587224274deb66
I20260505 07:54:18.421739 778 raft_consensus.cc:2468] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:18.428812 754 tablet_service.cc:1558] Processing DeleteTablet for tablet 73042a7ff5fb43f7a781b4f98f4f4020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.431680 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet ff7ed926a3ff4e7993864de734ca3c50 (table default.Table_1b644b185b194940a8a8a90595ae9e90 [id=8e18fbfbd36340fabddf08dc99fea437]) successfully deleted
I20260505 07:54:18.432201 2276 ts_tablet_manager.cc:1950] T b256d072933b4566b5587224274deb66 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:18.432567 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 190) took 7771 ms. Trace:
I20260505 07:54:18.432842 2276 rpcz_store.cc:276] 0505 07:54:10.661253 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.661483 (+ 230us) service_pool.cc:224] Handling call
0505 07:54:17.865147 (+7203664us) ts_tablet_manager.cc:1151] Deleting tablet b256d072933b4566b5587224274deb66
0505 07:54:17.865157 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.290455 (+425298us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.395121 (+104666us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.432551 (+ 37430us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7203531}
I20260505 07:54:18.436964 750 tablet_service.cc:1511] Processing CreateTablet for tablet 48890c946d3542948f8c53a552372867 (DEFAULT_TABLE table=default.Table_6125fe0ed26848bfbb9ed9d193944fa1 [id=d8c37876cf824007b39ebaef7f46f2ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.440039 3004 raft_consensus.cc:493] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:18.441895 3004 raft_consensus.cc:515] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.442560 3004 leader_election.cc:290] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.443748 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
W20260505 07:54:18.444106 718 leader_election.cc:343] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 46274a44a06b4306b5dac43c66c85a80
I20260505 07:54:18.409898 1026 tablet_service.cc:1558] Processing DeleteTablet for tablet 54e4f7f895d943639653c8c32724baf2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:06 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.410333 1031 tablet_service.cc:1558] Processing DeleteTablet for tablet a32eeaca77c1475f879465cb1046543b with delete_type TABLET_DATA_DELETED (Sending initial creation of tablet) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.410562 1034 tablet_service.cc:1558] Processing DeleteTablet for tablet b256d072933b4566b5587224274deb66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:09 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.410794 1019 tablet_service.cc:1558] Processing DeleteTablet for tablet bc9ae6404dd64bd8bf4898768e94e4f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.411023 1023 tablet_service.cc:1558] Processing DeleteTablet for tablet f92c53c68a4b4e319e98b84128e8e6a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.411231 1025 tablet_service.cc:1558] Processing DeleteTablet for tablet b5a0cbf9c3d74cea92920189da332c1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.411451 1020 tablet_service.cc:1558] Processing DeleteTablet for tablet feeafd614b3e4281a8e45b950e699b63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.411701 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 26f3544e74d941d286a2a0317b941ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.411921 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 06904a54b1c5463da31af0bc00fb606f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.412173 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 141665fcd59048bf926acd0287f1853e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.412269 1033 tablet_service.cc:1558] Processing DeleteTablet for tablet 62d4bc3154704ba2bfde83bca10f812d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.412472 1028 tablet_service.cc:1558] Processing DeleteTablet for tablet 0db162dfad2448e8ac428b3d3fbe17f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.412685 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 067c32ecccb44bca8bf9cc3f6c229326 with delete_type TABLET_DATA_DELETED (replaced by tablet d5083f98e8e14ccf99a19cdf2bc59018) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.412909 1018 tablet_service.cc:1558] Processing DeleteTablet for tablet a7d8265c05b74b28959d0de41fad1fde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.413125 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet 17327cee69a24e0cb493ff55f948bf6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.413328 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 73042a7ff5fb43f7a781b4f98f4f4020 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.413563 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet 23fa401bb4634022bca3c6b23421e4d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.413791 1016 tablet_service.cc:1558] Processing DeleteTablet for tablet bf76e868bbcb44b0b1887c84dd27f093 with delete_type TABLET_DATA_DELETED (replaced by tablet c837877946604ef29f95dac9dd449bf3) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.414111 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 17b4e886bebc4ab6a3c95290d18b4229 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.451351 2268 raft_consensus.cc:2272] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.452018 2268 ts_tablet_manager.cc:1916] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.456169 2276 tablet_replica.cc:333] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:18.456151 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet b256d072933b4566b5587224274deb66 (table default.Table_6cb216f92da74a78acd72a6132b94c1a [id=c0306ceddf024a1589f83f390b38c602]) successfully deleted
I20260505 07:54:18.456398 2276 raft_consensus.cc:2243] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.456576 2276 raft_consensus.cc:2272] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.457059 2276 ts_tablet_manager.cc:1916] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:18.414450 991 leader_election.cc:343] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 60772a9697ba417b879dc6025eeb8eec
I20260505 07:54:18.437477 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "898ebd1a70984cc1b9a71009b1070bfe" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.437696 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a85b65b1904bd6b793af48f71ed726" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.437878 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1506cd31fe364e328867bbf70e36845e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:18.467543 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 53100us
I20260505 07:54:18.467896 991 leader_election.cc:304] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:18.468451 3254 raft_consensus.cc:2804] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.468628 3254 raft_consensus.cc:697] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:18.468918 3254 consensus_queue.cc:237] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:18.440071 717 leader_election.cc:343] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: a7fe9c7199cc4039ae877dc3d4aab17e
I20260505 07:54:18.470402 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 72da22d790624b07bfca6f6b6301c925 (table default.Table_201f8409684e49be8bc5cd5249606250 [id=5a2b5d20e7a34b58b6d3fa93a255b2e1]) successfully deleted
W20260505 07:54:18.470702 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f88a5c0d0e2c436c8d8b6afc1f3c2edf because the tablet was not found. No further retry: Not found: Tablet not found: f88a5c0d0e2c436c8d8b6afc1f3c2edf
W20260505 07:54:18.471238 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet b12d2d2b866348d781209fa787da7d7e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b12d2d2b866348d781209fa787da7d7e already in progress: creating tablet
I20260505 07:54:18.473008 3145 raft_consensus.cc:493] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.473122 3145 raft_consensus.cc:515] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.473628 3145 leader_election.cc:290] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:18.474177 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d86214559d014f028acc5a9691efc9ca" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:18.474314 988 leader_election.cc:343] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.474398 988 leader_election.cc:304] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.474556 3255 raft_consensus.cc:2749] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.469941 717 leader_election.cc:304] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
W20260505 07:54:18.474795 854 leader_election.cc:343] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.474845 3004 raft_consensus.cc:2749] T a7fe9c7199cc4039ae877dc3d4aab17e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:18.475193 717 leader_election.cc:343] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: e29ba55a846548679cb3b86305d0d6ed
I20260505 07:54:18.475358 717 leader_election.cc:304] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:18.475626 3004 raft_consensus.cc:2749] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:18.475677 3145 raft_consensus.cc:493] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:18.475854 988 leader_election.cc:343] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:54:18.475809 3145 raft_consensus.cc:515] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:54:18.476011 717 leader_election.cc:343] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: Tablet not RUNNING: STOPPED
I20260505 07:54:18.476137 717 leader_election.cc:304] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:18.476135 988 leader_election.cc:304] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.476332 3202 raft_consensus.cc:2749] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.476380 3004 raft_consensus.cc:2749] T 72da22d790624b07bfca6f6b6301c925 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.476763 988 leader_election.cc:304] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:18.477531 3202 raft_consensus.cc:2804] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:18.477640 3202 raft_consensus.cc:493] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:18.477758 3202 raft_consensus.cc:3060] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.481958 3202 raft_consensus.cc:515] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.482448 3202 leader_election.cc:290] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.485091 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 5683d7366de84653ace56f0edcd3818c with delete_type TABLET_DATA_DELETED (replaced by tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.485456 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 9044763bed1a41739ecce4def4cca1fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.485813 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 8a60c015136246a3b02f70254c0bb67f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.486117 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 25072055f5534078adf5b5025614f951 with delete_type TABLET_DATA_DELETED (replaced by tablet 52e2db0240e04d6d8e948ebaedcc7d24) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.486416 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet d896e290167b4d0081e8e8b831c8ae2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.486722 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 880e326cbea74233a0ec5abb31f5b0d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.487025 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 323269e708964848a94c64e6a781a860 with delete_type TABLET_DATA_DELETED (replaced by tablet e010f23c4df647ab863e03c3cfed7898) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.487250 1350 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_aa752db71710464497d37acff57357ef"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:18.487382 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet abaa596871bb4a82aefc317932e91d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:05 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.487679 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 27bacc97d7534284970f218cf68ac75b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
W20260505 07:54:18.487867 1350 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aa752db71710464497d37acff57357ef in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.487941 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet 202a237ca580482c98803683a77d96d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.488443 1176 tablet_bootstrap.cc:492] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.488560 1176 ts_tablet_manager.cc:1403] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.142s user 0.009s sys 0.013s
I20260505 07:54:18.489010 3145 leader_election.cc:290] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:18.489593 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "411025888b55405586b128d5bc0f2021" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:18.490069 854 leader_election.cc:343] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.490430 1349 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:33140 (request call id 0) took 4533 ms. Trace:
I20260505 07:54:18.490597 1176 raft_consensus.cc:359] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.490741 1176 raft_consensus.cc:385] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.490811 1176 raft_consensus.cc:740] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.490594 1349 rpcz_store.cc:276] 0505 07:54:13.956684 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.423443 (+4466759us) service_pool.cc:224] Handling call
0505 07:54:18.490255 (+ 66812us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639df215 00007fd65a47c747 00007fd65a47c475 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:18.490388 (+ 133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152929664}
I20260505 07:54:18.491048 1176 consensus_queue.cc:260] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:18.491333 717 consensus_peers.cc:597] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 141665fcd59048bf926acd0287f1853e. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:18.491360 1349 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6235f4345c354a6daa96d2ece798f82e" } modify_external_catalogs: true
I20260505 07:54:18.491559 1349 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_6235f4345c354a6daa96d2ece798f82e" } modify_external_catalogs: true
W20260505 07:54:18.491622 717 leader_election.cc:343] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: ba616702dc6040bb805bb3ffc8a8a871
I20260505 07:54:18.491832 717 leader_election.cc:304] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:18.492429 1176 ts_tablet_manager.cc:1434] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:18.492853 1353 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_9cfdbaa4e1834e3fad8d09804f626d78" } modify_external_catalogs: true
I20260505 07:54:18.492991 1353 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_9cfdbaa4e1834e3fad8d09804f626d78" } modify_external_catalogs: true
I20260505 07:54:18.493446 1358 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_a2dcfcc5bec246db8d5426a08af47ed9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:18.494017 1358 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2dcfcc5bec246db8d5426a08af47ed9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:18.494952 1318 connection.cc:463] server connection from 127.0.0.1:33140 torn down before Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:33140 (request call id 0) could send its response
W20260505 07:54:18.497189 856 leader_election.cc:343] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.497304 856 leader_election.cc:304] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.497622 3145 raft_consensus.cc:2749] T 898ebd1a70984cc1b9a71009b1070bfe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.498523 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bf49b787ef4de095448751464708a4" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:18.498679 775 raft_consensus.cc:3060] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.501303 775 raft_consensus.cc:2468] T e8bf49b787ef4de095448751464708a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:18.501606 1355 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_21673305a26a4342a0be11d98e875447" } modify_external_catalogs: true
I20260505 07:54:18.501799 1355 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_21673305a26a4342a0be11d98e875447" } modify_external_catalogs: true
I20260505 07:54:18.505175 3004 raft_consensus.cc:2749] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:18.505595 717 leader_election.cc:304] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:18.505864 3004 raft_consensus.cc:2804] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:18.505951 3004 raft_consensus.cc:493] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:18.506022 3004 raft_consensus.cc:3060] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:18.506678 1181 tablet_bootstrap.cc:492] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:18.506791 1181 ts_tablet_manager.cc:1403] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.118s user 0.003s sys 0.000s
I20260505 07:54:18.507752 1181 raft_consensus.cc:359] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.507897 1181 raft_consensus.cc:385] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.507951 1181 raft_consensus.cc:740] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.508095 1181 consensus_queue.cc:260] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.508442 3004 raft_consensus.cc:515] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.508831 1181 ts_tablet_manager.cc:1434] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:18.509327 1181 tablet_bootstrap.cc:492] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:18.509622 3004 leader_election.cc:290] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.510913 1181 tablet_bootstrap.cc:654] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:18.498050 856 leader_election.cc:343] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.514734 856 leader_election.cc:304] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
W20260505 07:54:18.515115 856 leader_election.cc:343] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.515223 856 leader_election.cc:304] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.516539 3145 raft_consensus.cc:2749] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.516698 3145 raft_consensus.cc:2749] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:18.520100 718 leader_election.cc:343] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: df1b62a30b2842b897a04c793bb5e7ce
I20260505 07:54:18.530644 2276 ts_tablet_manager.cc:1929] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.530748 2276 log.cc:1199] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/17b4e886bebc4ab6a3c95290d18b4229
I20260505 07:54:18.511303 1030 tablet_service.cc:1558] Processing DeleteTablet for tablet e8bf49b787ef4de095448751464708a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.512694 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d86214559d014f028acc5a9691efc9ca" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.512992 1016 tablet_service.cc:1558] Processing DeleteTablet for tablet 4ba2cc218300472cb5fd6dc8cfb7811c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.513260 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "411025888b55405586b128d5bc0f2021" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.513424 1022 tablet_service.cc:1558] Processing DeleteTablet for tablet e6a5f823db6143319d743d363a77ea47 with delete_type TABLET_DATA_DELETED (replaced by tablet 90612f0afb4949f491e625c61e40d247) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.518374 3173 raft_consensus.cc:493] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.518725 1035 tablet_service.cc:1558] Processing DeleteTablet for tablet 003cd4472b424aacbba8c5f89900e228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.518975 1032 tablet_service.cc:1558] Processing DeleteTablet for tablet 12501fdf2cc5491cb1a06db490e70810 with delete_type TABLET_DATA_DELETED (replaced by tablet d622612bdc404727a27b83769bf39738) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.519733 1018 tablet_service.cc:1558] Processing DeleteTablet for tablet 78a6470720984761a8606abeafbef992 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.520222 1017 tablet_service.cc:1558] Processing DeleteTablet for tablet 91d57f5bb6e847e6b099134d1382b730 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:10 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.520458 1028 tablet_service.cc:1558] Processing DeleteTablet for tablet b0b9e8eb767a4eceb583ff9b47000d58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:01 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.520713 1033 tablet_service.cc:1558] Processing DeleteTablet for tablet d622612bdc404727a27b83769bf39738 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:57 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.520925 1021 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f1feb0387bc4afead72346398dbe9e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.521137 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 03f31efbc98c484c950c70d5e84d2136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:55 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.521848 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 9959be9f83c2444282d012b61eef49b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:54 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.532277 3173 raft_consensus.cc:515] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.533152 3173 leader_election.cc:290] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.527997 988 leader_election.cc:304] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.533674 3173 raft_consensus.cc:2804] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.533819 3173 raft_consensus.cc:697] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:18.534060 3173 consensus_queue.cc:237] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:18.542172 856 leader_election.cc:343] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.542279 856 leader_election.cc:304] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.542604 3144 raft_consensus.cc:2749] T d86214559d014f028acc5a9691efc9ca P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:18.541806 1357 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.3:45737 (request call id 163) took 2438 ms (client timeout 1000 ms). Trace:
W20260505 07:54:18.543085 856 leader_election.cc:343] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.543165 856 leader_election.cc:304] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.543439 3144 raft_consensus.cc:2749] T 411025888b55405586b128d5bc0f2021 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.544066 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225c56ac920b448c89c38d6fb3c1bdef" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:18.545050 988 leader_election.cc:343] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.546579 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4a0ef596e66462a8927773dc86b2546. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:18.543433 1357 rpcz_store.cc:269] 0505 07:54:16.102349 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.400346 (+297997us) service_pool.cc:224] Handling call
0505 07:54:18.540617 (+2140271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99541,"reported_tablets":271}
I20260505 07:54:18.551792 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08dafc2271dc49cfaf55915ef0338ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.552569 3261 raft_consensus.cc:493] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.552688 3261 raft_consensus.cc:515] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.553252 3261 leader_election.cc:290] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.555850 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a54d94d8a888483681858398f667a370. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:18.556394 991 leader_election.cc:343] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: e8bf49b787ef4de095448751464708a4
I20260505 07:54:18.557523 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225c56ac920b448c89c38d6fb3c1bdef" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:18.557696 918 raft_consensus.cc:2468] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:18.559511 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 260a8d41de464e429491ada4bfe2c761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.560061 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 184) took 9005 ms. Trace:
I20260505 07:54:18.560153 2447 rpcz_store.cc:276] 0505 07:54:09.554762 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.554838 (+ 76us) service_pool.cc:224] Handling call
0505 07:54:18.378029 (+8823191us) ts_tablet_manager.cc:1151] Deleting tablet 60772a9697ba417b879dc6025eeb8eec
0505 07:54:18.378038 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.382726 (+ 4688us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.397891 (+ 15165us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.560038 (+162147us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8818689}
I20260505 07:54:18.560483 2447 tablet_replica.cc:333] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:18.560686 2447 raft_consensus.cc:2243] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.560845 2447 raft_consensus.cc:2272] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.560948 991 leader_election.cc:304] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.561177 3202 raft_consensus.cc:2804] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:18.561239 3202 raft_consensus.cc:493] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:18.561291 3202 raft_consensus.cc:3060] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.561367 2447 ts_tablet_manager.cc:1916] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.564335 3202 raft_consensus.cc:515] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.564863 3202 leader_election.cc:290] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.565388 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225c56ac920b448c89c38d6fb3c1bdef" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:18.565899 988 leader_election.cc:343] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.566509 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70051011024f4971bd0166d335d32393" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.566660 782 raft_consensus.cc:2468] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:18.567013 2276 ts_tablet_manager.cc:1950] T 17b4e886bebc4ab6a3c95290d18b4229 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:18.567337 988 leader_election.cc:304] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.567438 2447 ts_tablet_manager.cc:1929] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.567520 2447 log.cc:1199] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/26f3544e74d941d286a2a0317b941ab6
I20260505 07:54:18.567565 3202 raft_consensus.cc:2804] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:18.567655 3202 raft_consensus.cc:493] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:18.567711 3202 raft_consensus.cc:3060] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.567919 2447 ts_tablet_manager.cc:1950] T 26f3544e74d941d286a2a0317b941ab6 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:18.568539 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 183) took 9014 ms. Trace:
I20260505 07:54:18.568624 2447 rpcz_store.cc:276] 0505 07:54:09.554361 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.554405 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:18.560436 (+9006031us) ts_tablet_manager.cc:1151] Deleting tablet 26f3544e74d941d286a2a0317b941ab6
0505 07:54:18.560447 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.564531 (+ 4084us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.567398 (+ 2867us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.568523 (+ 1125us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9001036}
I20260505 07:54:18.568912 2447 tablet_replica.cc:333] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:18.569080 2447 raft_consensus.cc:2243] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.569214 2447 raft_consensus.cc:2272] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.569707 2447 ts_tablet_manager.cc:1916] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.571373 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48890c946d3542948f8c53a552372867. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.571498 1176 tablet_bootstrap.cc:492] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.572845 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225c56ac920b448c89c38d6fb3c1bdef" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:18.573066 912 raft_consensus.cc:3060] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.576926 1181 tablet_bootstrap.cc:492] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:18.558522 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70051011024f4971bd0166d335d32393" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:18.577204 918 raft_consensus.cc:2468] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:18.578136 912 raft_consensus.cc:2468] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:18.578509 1181 ts_tablet_manager.cc:1403] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.069s user 0.002s sys 0.001s
I20260505 07:54:18.579370 1181 raft_consensus.cc:359] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.579574 1181 raft_consensus.cc:385] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.579697 1181 raft_consensus.cc:740] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.579938 1181 consensus_queue.cc:260] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.580740 1181 ts_tablet_manager.cc:1434] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:18.581432 1181 tablet_bootstrap.cc:492] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:18.581763 2447 ts_tablet_manager.cc:1929] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.581732 991 leader_election.cc:304] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.599604 3144 raft_consensus.cc:493] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.599741 3144 raft_consensus.cc:515] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.600312 3144 leader_election.cc:290] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:18.603981 3254 raft_consensus.cc:2804] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.604101 3254 raft_consensus.cc:697] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:18.604537 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676e2109e59749b58931d597ce1beb9c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.604321 3254 consensus_queue.cc:237] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.613123 2447 log.cc:1199] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/1852f7f4ff594639908b03730b68fe86
I20260505 07:54:18.593741 3004 raft_consensus.cc:493] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.600944 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676e2109e59749b58931d597ce1beb9c" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:18.601967 718 consensus_peers.cc:597] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: SHUTDOWN. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:18.616217 3004 raft_consensus.cc:515] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.616959 3004 leader_election.cc:290] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.617553 2447 ts_tablet_manager.cc:1950] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:18.619056 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 163) took 9123 ms. Trace:
I20260505 07:54:18.619567 1357 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_bffe918db391471ba1407a66a5b7c240" } modify_external_catalogs: true
I20260505 07:54:18.619818 2447 rpcz_store.cc:276] 0505 07:54:09.495480 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.495641 (+ 161us) service_pool.cc:224] Handling call
0505 07:54:18.568875 (+9073234us) ts_tablet_manager.cc:1151] Deleting tablet 1852f7f4ff594639908b03730b68fe86
0505 07:54:18.568883 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.578834 (+ 9951us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.581707 (+ 2873us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.619037 (+ 37330us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9008839}
I20260505 07:54:18.620371 2447 tablet_replica.cc:333] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
W20260505 07:54:18.621614 856 leader_election.cc:343] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:18.621850 854 leader_election.cc:343] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.622061 854 leader_election.cc:304] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.622908 3144 raft_consensus.cc:2749] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.623284 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbfb2345a1a4b218c2dddbae6667e47" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:18.623509 907 raft_consensus.cc:2393] T 9cbfb2345a1a4b218c2dddbae6667e47 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:18.624506 1181 tablet_bootstrap.cc:654] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.625088 2447 raft_consensus.cc:2243] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.625332 2447 raft_consensus.cc:2272] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.625963 2447 ts_tablet_manager.cc:1916] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.624954 3202 raft_consensus.cc:515] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.629305 3202 leader_election.cc:290] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:18.587275 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 73912.4us
I20260505 07:54:18.629722 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70051011024f4971bd0166d335d32393" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:18.629897 918 raft_consensus.cc:3060] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.630019 3004 consensus_queue.cc:1059] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260505 07:54:18.630266 3133 raft_consensus.cc:3055] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Stepping down as leader of term 1
I20260505 07:54:18.630339 3133 raft_consensus.cc:740] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:18.630560 3133 consensus_queue.cc:260] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.630949 3133 raft_consensus.cc:3060] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
W20260505 07:54:18.631558 717 consensus_peers.cc:597] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f92c53c68a4b4e319e98b84128e8e6a4. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:18.631810 717 consensus_peers.cc:597] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:18.631963 717 leader_election.cc:343] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 1852f7f4ff594639908b03730b68fe86
I20260505 07:54:18.632055 717 leader_election.cc:304] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
W20260505 07:54:18.632217 717 raft_consensus.cc:2686] T 1852f7f4ff594639908b03730b68fe86 P eac7e1e8caba43cdb4df80cbb80e891c: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260505 07:54:18.634228 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70051011024f4971bd0166d335d32393" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:18.634369 775 raft_consensus.cc:3060] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.636767 775 raft_consensus.cc:2468] T 70051011024f4971bd0166d335d32393 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:18.634860 918 raft_consensus.cc:2468] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:18.637804 1172 tablet_bootstrap.cc:492] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:18.637945 1172 ts_tablet_manager.cc:1403] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.429s user 0.028s sys 0.032s
W20260505 07:54:18.642433 717 consensus_peers.cc:597] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 17b4e886bebc4ab6a3c95290d18b4229. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:18.643303 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 60772a9697ba417b879dc6025eeb8eec (table default.Table_ee5371f299e44106859f3663fe6052b7 [id=a4d53a8c6539419daba67a153292bb0c]) successfully deleted
I20260505 07:54:18.643764 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 26f3544e74d941d286a2a0317b941ab6 (table default.Table_82c755fca0b64337a44a63876e04fa68 [id=ded6ca9533da4aa197bbd8c9b4a72066]) successfully deleted
I20260505 07:54:18.644305 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 1852f7f4ff594639908b03730b68fe86 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]) successfully deleted
I20260505 07:54:18.644935 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df1b62a30b2842b897a04c793bb5e7ce" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:18.645074 1047 raft_consensus.cc:3060] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:18.646332 1172 raft_consensus.cc:359] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.646495 1172 raft_consensus.cc:385] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.646544 1172 raft_consensus.cc:740] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.646714 1172 consensus_queue.cc:260] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.647756 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cbfb2345a1a4b218c2dddbae6667e47" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.650216 1172 ts_tablet_manager.cc:1434] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260505 07:54:18.650502 1172 tablet_bootstrap.cc:492] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:18.630107 2268 ts_tablet_manager.cc:1929] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.650796 2268 log.cc:1199] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/a9b7c6e9463342a5a73af12da4447f89
I20260505 07:54:18.651189 2268 ts_tablet_manager.cc:1950] T a9b7c6e9463342a5a73af12da4447f89 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:18.651245 1047 raft_consensus.cc:2468] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 2.
I20260505 07:54:18.651850 1172 tablet_bootstrap.cc:654] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.652009 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 259) took 7435 ms. Trace:
I20260505 07:54:18.652348 2268 rpcz_store.cc:276] 0505 07:54:11.216957 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.217024 (+ 67us) service_pool.cc:224] Handling call
0505 07:54:18.401710 (+7184686us) ts_tablet_manager.cc:1151] Deleting tablet a9b7c6e9463342a5a73af12da4447f89
0505 07:54:18.401717 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.455045 (+ 53328us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.630055 (+175010us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.651958 (+ 21903us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7184530}
I20260505 07:54:18.652680 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 191) took 7911 ms. Trace:
I20260505 07:54:18.652796 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 260) took 7435 ms. Trace:
I20260505 07:54:18.652762 2276 rpcz_store.cc:276] 0505 07:54:10.741595 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.742353 (+ 758us) service_pool.cc:224] Handling call
0505 07:54:18.456116 (+7713763us) ts_tablet_manager.cc:1151] Deleting tablet 17b4e886bebc4ab6a3c95290d18b4229
0505 07:54:18.456126 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.504340 (+ 48214us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.530590 (+ 26250us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.652633 (+122043us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7713579}
I20260505 07:54:18.652895 2268 rpcz_store.cc:276] 0505 07:54:11.217639 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.217697 (+ 58us) service_pool.cc:224] Handling call
0505 07:54:18.652672 (+7434975us) ts_tablet_manager.cc:1151] Deleting tablet 85d485ae2d1d4d76a6a5d988cec8291a
0505 07:54:18.652681 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.652736 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7434843}
W20260505 07:54:18.653211 717 leader_election.cc:343] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 46274a44a06b4306b5dac43c66c85a80
I20260505 07:54:18.653311 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 262) took 7434 ms. Trace:
I20260505 07:54:18.653374 717 leader_election.cc:304] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:18.653431 2268 rpcz_store.cc:276] 0505 07:54:11.218836 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.218909 (+ 73us) service_pool.cc:224] Handling call
0505 07:54:18.653242 (+7434333us) ts_tablet_manager.cc:1151] Deleting tablet 4662d4a3d88d4c2883a35048d6e11eb9
0505 07:54:18.653251 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.653298 (+ 47us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7434179}
I20260505 07:54:18.653548 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]) successfully deleted
I20260505 07:54:18.653733 2268 tablet_replica.cc:333] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:18.653981 2268 raft_consensus.cc:2243] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.654178 2268 raft_consensus.cc:2272] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.654608 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 192) took 7901 ms. Trace:
I20260505 07:54:18.654698 2276 rpcz_store.cc:276] 0505 07:54:10.753178 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.754061 (+ 883us) service_pool.cc:224] Handling call
0505 07:54:18.654374 (+7900313us) ts_tablet_manager.cc:1151] Deleting tablet 91d57f5bb6e847e6b099134d1382b730
0505 07:54:18.654384 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.654594 (+ 210us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7899573}
W20260505 07:54:18.655055 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 91d57f5bb6e847e6b099134d1382b730 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91d57f5bb6e847e6b099134d1382b730 already in progress: creating tablet
I20260505 07:54:18.655162 2276 tablet_replica.cc:333] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:18.655346 2276 raft_consensus.cc:2243] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.655483 2276 raft_consensus.cc:2272] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.655977 2276 ts_tablet_manager.cc:1916] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.656626 3133 raft_consensus.cc:2749] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:18.638127 991 leader_election.cc:304] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:18.658020 3261 raft_consensus.cc:2804] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.658172 3261 raft_consensus.cc:697] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:18.658422 3261 consensus_queue.cc:237] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.660940 717 leader_election.cc:304] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:18.662143 3133 raft_consensus.cc:3060] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.664845 1176 tablet_bootstrap.cc:654] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.666859 3133 raft_consensus.cc:2749] T 9cbfb2345a1a4b218c2dddbae6667e47 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.667512 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet a9b7c6e9463342a5a73af12da4447f89 (table default.Table_0393e87d0a314ee898b11935e2f6666d [id=ab93bfbabbad4fad893f7a6dd595540f]) successfully deleted
W20260505 07:54:18.668164 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 85d485ae2d1d4d76a6a5d988cec8291a because the tablet was not found. No further retry: Not found: Tablet not found: 85d485ae2d1d4d76a6a5d988cec8291a
W20260505 07:54:18.668670 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 4662d4a3d88d4c2883a35048d6e11eb9 because the tablet was not found. No further retry: Not found: Tablet not found: 4662d4a3d88d4c2883a35048d6e11eb9
I20260505 07:54:18.670222 717 leader_election.cc:304] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:18.673149 3133 raft_consensus.cc:2804] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:18.674275 3133 raft_consensus.cc:697] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 2 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:18.674564 3133 consensus_queue.cc:237] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.683017 3144 raft_consensus.cc:493] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.683139 3144 raft_consensus.cc:515] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.683861 3144 leader_election.cc:290] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:18.684286 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae829daad64545e687a9528e5c38b9ed" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:18.684865 854 leader_election.cc:343] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.685366 2831 raft_consensus.cc:493] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:18.685493 2831 raft_consensus.cc:515] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.686044 2831 leader_election.cc:290] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.686335 2268 ts_tablet_manager.cc:1916] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:18.686918 718 leader_election.cc:343] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 472441bd29704be59aabb0f7a4964bb0
I20260505 07:54:18.687300 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae829daad64545e687a9528e5c38b9ed" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:18.689713 856 leader_election.cc:343] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.689833 856 leader_election.cc:304] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.690433 3144 raft_consensus.cc:2749] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.689347 2447 ts_tablet_manager.cc:1929] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:18.691344 1181 tablet_bootstrap.cc:492] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:18.692499 2447 log.cc:1199] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/7219f9896fd94187ae714635077110af
I20260505 07:54:18.692685 1181 ts_tablet_manager.cc:1403] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.111s user 0.003s sys 0.000s
I20260505 07:54:18.693102 2447 ts_tablet_manager.cc:1950] T 7219f9896fd94187ae714635077110af P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:18.693468 1181 raft_consensus.cc:359] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.694538 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 192) took 9052 ms. Trace:
I20260505 07:54:18.697445 1181 raft_consensus.cc:385] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.697599 1181 raft_consensus.cc:740] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.698290 1181 consensus_queue.cc:260] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.701025 1181 ts_tablet_manager.cc:1434] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20260505 07:54:18.704066 1181 tablet_bootstrap.cc:492] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:18.705909 1181 tablet_bootstrap.cc:654] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.713956 1353 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet fc0fd88889b54aac88c4f18c0abe7a43
I20260505 07:54:18.714669 1353 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_041db4f8d9684d8e97ba6def427d554d" } new_table_name: "default.Table_53d2dcb1d87247a6add98d671bfa470b" modify_external_catalogs: true
I20260505 07:54:18.717900 1355 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 0e9a98243da14d43bc649bb19aed4f0c
I20260505 07:54:18.718309 896 tablet_service.cc:1558] Processing DeleteTablet for tablet fc0fd88889b54aac88c4f18c0abe7a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:18.718357 2447 rpcz_store.cc:276] 0505 07:54:09.642355 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.642422 (+ 67us) service_pool.cc:224] Handling call
0505 07:54:18.620327 (+8977905us) ts_tablet_manager.cc:1151] Deleting tablet 7219f9896fd94187ae714635077110af
0505 07:54:18.620338 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.655666 (+ 35328us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.689292 (+ 33626us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.694518 (+ 5226us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8952027}
I20260505 07:54:18.714538 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet fc0fd88889b54aac88c4f18c0abe7a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.718727 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 193) took 9076 ms. Trace:
I20260505 07:54:18.718812 2447 rpcz_store.cc:276] 0505 07:54:09.642726 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.642788 (+ 62us) service_pool.cc:224] Handling call
0505 07:54:18.718622 (+9075834us) ts_tablet_manager.cc:1151] Deleting tablet 25072055f5534078adf5b5025614f951
0505 07:54:18.718631 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.718713 (+ 82us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9050285}
I20260505 07:54:18.719046 2447 tablet_replica.cc:333] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:18.719256 2447 raft_consensus.cc:2243] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.719403 2447 raft_consensus.cc:2272] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:18.712334 717 leader_election.cc:343] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 472441bd29704be59aabb0f7a4964bb0
I20260505 07:54:18.723183 717 leader_election.cc:304] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:18.723676 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 7219f9896fd94187ae714635077110af (table default.Table_ad09d6b42bc344c6ba96f4322d75601f [id=d82da77797e14074b53f00e8e741744c]) successfully deleted
I20260505 07:54:18.715404 1176 tablet_bootstrap.cc:492] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.724078 750 tablet_service.cc:1558] Processing DeleteTablet for tablet fc0fd88889b54aac88c4f18c0abe7a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.724164 1176 ts_tablet_manager.cc:1403] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.153s user 0.013s sys 0.000s
I20260505 07:54:18.724249 748 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a98243da14d43bc649bb19aed4f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.724845 1176 raft_consensus.cc:359] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.725004 1176 raft_consensus.cc:385] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.725061 1176 raft_consensus.cc:740] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.725183 2268 ts_tablet_manager.cc:1929] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.725270 1176 consensus_queue.cc:260] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.725515 2268 log.cc:1199] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/f842d8ac6b2142ffa69945c6087be34b
I20260505 07:54:18.725524 2831 raft_consensus.cc:2749] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:18.725791 1176 ts_tablet_manager.cc:1434] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:18.726047 1176 tablet_bootstrap.cc:492] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.727630 1176 tablet_bootstrap.cc:654] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.727842 2268 ts_tablet_manager.cc:1950] T f842d8ac6b2142ffa69945c6087be34b P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:18.728617 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 239) took 7682 ms. Trace:
I20260505 07:54:18.728751 2276 ts_tablet_manager.cc:1929] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.728739 2268 rpcz_store.cc:276] 0505 07:54:11.045716 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.046592 (+ 876us) service_pool.cc:224] Handling call
0505 07:54:18.653698 (+7607106us) ts_tablet_manager.cc:1151] Deleting tablet f842d8ac6b2142ffa69945c6087be34b
0505 07:54:18.653704 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.704613 (+ 50909us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.724936 (+ 20323us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.728597 (+ 3661us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7235394}
I20260505 07:54:18.729373 2447 ts_tablet_manager.cc:1916] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.729426 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 276) took 6697 ms. Trace:
I20260505 07:54:18.731743 2831 raft_consensus.cc:493] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:18.731760 2268 rpcz_store.cc:276] 0505 07:54:12.031770 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.032043 (+ 273us) service_pool.cc:224] Handling call
0505 07:54:18.729327 (+6697284us) ts_tablet_manager.cc:1151] Deleting tablet f57e14ee70584d258c83ff1b01189713
0505 07:54:18.729337 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.729412 (+ 75us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6697129}
I20260505 07:54:18.731860 2831 raft_consensus.cc:515] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.732399 2831 leader_election.cc:290] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:18.733052 718 leader_election.cc:343] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: d7e0c8ad716e443588d4ae3aa28ed4bf
I20260505 07:54:18.718228 1357 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_bffe918db391471ba1407a66a5b7c240" } modify_external_catalogs: true
I20260505 07:54:18.733346 1172 tablet_bootstrap.cc:492] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:18.733496 1172 ts_tablet_manager.cc:1403] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.083s user 0.003s sys 0.002s
W20260505 07:54:18.733604 717 leader_election.cc:343] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: d7e0c8ad716e443588d4ae3aa28ed4bf
I20260505 07:54:18.733850 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]) successfully deleted
W20260505 07:54:18.734144 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f57e14ee70584d258c83ff1b01189713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f57e14ee70584d258c83ff1b01189713 already in progress: creating tablet
I20260505 07:54:18.734226 1172 raft_consensus.cc:359] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.734392 1172 raft_consensus.cc:385] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.734474 1172 raft_consensus.cc:740] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.734756 1172 consensus_queue.cc:260] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:54:18.734954 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 25072055f5534078adf5b5025614f951 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 25072055f5534078adf5b5025614f951 already in progress: creating tablet
I20260505 07:54:18.736045 1172 ts_tablet_manager.cc:1434] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:18.736724 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_c42ce47a2e564486bb790f681399d8e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:18.736979 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a98243da14d43bc649bb19aed4f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.737165 1172 tablet_bootstrap.cc:492] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:18.737244 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c42ce47a2e564486bb790f681399d8e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.733713 717 leader_election.cc:304] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:18.737973 2831 raft_consensus.cc:2749] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:18.738713 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 0e9a98243da14d43bc649bb19aed4f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:18.739323 2831 raft_consensus.cc:493] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.739445 1172 tablet_bootstrap.cc:654] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.739444 2831 raft_consensus.cc:515] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.740007 2831 leader_election.cc:290] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.740109 2276 log.cc:1199] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/f200abef13ac42698a9b96fc8af82ff0
I20260505 07:54:18.740520 2276 ts_tablet_manager.cc:1950] T f200abef13ac42698a9b96fc8af82ff0 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:18.740521 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36a0f68fc2b4adda04ec3b9db5f2d8e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:18.740727 918 raft_consensus.cc:2468] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:18.741091 718 leader_election.cc:304] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.741393 2831 raft_consensus.cc:2804] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:18.741463 2831 raft_consensus.cc:493] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:18.741534 2831 raft_consensus.cc:3060] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.743120 1357 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 139d5a2ab9594f6ea6042c1c4ba8a46a
I20260505 07:54:18.743921 896 tablet_service.cc:1558] Processing DeleteTablet for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:18.744410 2831 raft_consensus.cc:515] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.745223 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36a0f68fc2b4adda04ec3b9db5f2d8e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:18.745378 918 raft_consensus.cc:3060] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.745442 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_aeab2a6ff10d466f8126362debcb6db6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:18.745932 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aeab2a6ff10d466f8126362debcb6db6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:18.748250 918 raft_consensus.cc:2468] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:18.748975 718 leader_election.cc:304] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.749291 3004 raft_consensus.cc:2804] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.749393 3004 raft_consensus.cc:697] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:18.749473 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36a0f68fc2b4adda04ec3b9db5f2d8e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.749631 1051 raft_consensus.cc:2468] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:18.749594 3004 consensus_queue.cc:237] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.750691 748 tablet_service.cc:1558] Processing DeleteTablet for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.751863 1027 tablet_service.cc:1558] Processing DeleteTablet for tablet 139d5a2ab9594f6ea6042c1c4ba8a46a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:18.753168 2831 leader_election.cc:290] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.753582 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f36a0f68fc2b4adda04ec3b9db5f2d8e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:18.753763 1051 raft_consensus.cc:3060] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.754451 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 193) took 7998 ms. Trace:
I20260505 07:54:18.756809 1051 raft_consensus.cc:2468] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:18.761161 896 tablet_service.cc:1511] Processing CreateTablet for tablet 7839b616fc854974b0dca43aead118f4 (DEFAULT_TABLE table=default.Table_eebd7fed9d9d4b4c923a82aa7d302199 [id=ea983cbbaeca4fdb9c466690a750a2cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.761576 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7839b616fc854974b0dca43aead118f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.762544 1027 tablet_service.cc:1511] Processing CreateTablet for tablet 7839b616fc854974b0dca43aead118f4 (DEFAULT_TABLE table=default.Table_eebd7fed9d9d4b4c923a82aa7d302199 [id=ea983cbbaeca4fdb9c466690a750a2cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.762957 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7839b616fc854974b0dca43aead118f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.764626 899 tablet_service.cc:1511] Processing CreateTablet for tablet ee9f325dc993432fa07aaf099b76ca85 (DEFAULT_TABLE table=default.Table_7522f51dac38424194a5755cf5b39067 [id=c183d46e9c6e40d79c785d68a1f0dc1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.764972 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9f325dc993432fa07aaf099b76ca85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.764930 887 tablet_service.cc:1511] Processing CreateTablet for tablet 11200db846bc40d490845671366a00f5 (DEFAULT_TABLE table=default.Table_8ed3074214464ce3ae998a9580562f57 [id=8311571b866f48d68b8a915588bb1934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.765262 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11200db846bc40d490845671366a00f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.765424 885 tablet_service.cc:1511] Processing CreateTablet for tablet 2d393766f3f54c9db6a1fc4108bb0345 (DEFAULT_TABLE table=default.Table_7a2d015cb9324e77afad1280b0e8081c [id=27814470bfe94b1897f6d89c13710f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.765719 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d393766f3f54c9db6a1fc4108bb0345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.764659 892 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4858e05c744536a32c21e53ba2edf2 (DEFAULT_TABLE table=default.Table_153cefb63d3a4dfa894369fe70347fed [id=917e54c08fa34483b17d9d01efdfbefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.767951 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4858e05c744536a32c21e53ba2edf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.767918 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4858e05c744536a32c21e53ba2edf2 (DEFAULT_TABLE table=default.Table_153cefb63d3a4dfa894369fe70347fed [id=917e54c08fa34483b17d9d01efdfbefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.768289 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4858e05c744536a32c21e53ba2edf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.769882 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 277) took 6722 ms. Trace:
I20260505 07:54:18.769969 2268 rpcz_store.cc:276] 0505 07:54:12.047319 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.047380 (+ 61us) service_pool.cc:224] Handling call
0505 07:54:18.732216 (+6684836us) ts_tablet_manager.cc:1151] Deleting tablet b0d966bcd7f1419cbd090afc185ae78a
0505 07:54:18.732225 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.769865 (+ 37640us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6684683}
I20260505 07:54:18.770291 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 278) took 6716 ms. Trace:
I20260505 07:54:18.770351 2268 rpcz_store.cc:276] 0505 07:54:12.053659 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.053707 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:18.770194 (+6716487us) ts_tablet_manager.cc:1151] Deleting tablet b8a85b65b1904bd6b793af48f71ed726
0505 07:54:18.770202 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.770278 (+ 76us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6716346}
I20260505 07:54:18.770638 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 285) took 6358 ms. Trace:
I20260505 07:54:18.770691 2268 rpcz_store.cc:276] 0505 07:54:12.412453 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.412517 (+ 64us) service_pool.cc:224] Handling call
0505 07:54:18.770558 (+6358041us) ts_tablet_manager.cc:1151] Deleting tablet f840d5f58161468385d5c4f116b78c3d
0505 07:54:18.770565 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.770627 (+ 62us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6357011}
I20260505 07:54:18.770869 2268 tablet_replica.cc:333] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:18.771042 2268 raft_consensus.cc:2243] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:18.771306 2268 raft_consensus.cc:2272] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.771687 1021 tablet_service.cc:1511] Processing CreateTablet for tablet 11200db846bc40d490845671366a00f5 (DEFAULT_TABLE table=default.Table_8ed3074214464ce3ae998a9580562f57 [id=8311571b866f48d68b8a915588bb1934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.772068 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11200db846bc40d490845671366a00f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.774907 3144 raft_consensus.cc:493] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.775060 3144 raft_consensus.cc:515] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.775760 3144 leader_election.cc:290] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:18.754825 2276 rpcz_store.cc:276] 0505 07:54:10.755645 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.755889 (+ 244us) service_pool.cc:224] Handling call
0505 07:54:18.655121 (+7899232us) ts_tablet_manager.cc:1151] Deleting tablet f200abef13ac42698a9b96fc8af82ff0
0505 07:54:18.655131 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.698634 (+ 43503us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.728697 (+ 30063us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.754322 (+ 25625us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7899031}
I20260505 07:54:18.780237 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4fc18f6faa488882d9bffb30793cdd" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.780443 2276 tablet_replica.cc:333] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
W20260505 07:54:18.780798 856 leader_election.cc:343] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.781201 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet f200abef13ac42698a9b96fc8af82ff0 (table default.Table_157815480b75424885451ec833e3438d [id=ebd83e4e101d4282ab8aeec21c281c60]) successfully deleted
I20260505 07:54:18.781302 2276 raft_consensus.cc:2243] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:18.781613 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet b0d966bcd7f1419cbd090afc185ae78a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b0d966bcd7f1419cbd090afc185ae78a already in progress: creating tablet
W20260505 07:54:18.781858 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet b8a85b65b1904bd6b793af48f71ed726 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b8a85b65b1904bd6b793af48f71ed726 already in progress: creating tablet
I20260505 07:54:18.781857 2276 raft_consensus.cc:2272] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:18.782114 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f840d5f58161468385d5c4f116b78c3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f840d5f58161468385d5c4f116b78c3d already in progress: creating tablet
I20260505 07:54:18.782902 1033 tablet_service.cc:1511] Processing CreateTablet for tablet ee9f325dc993432fa07aaf099b76ca85 (DEFAULT_TABLE table=default.Table_7522f51dac38424194a5755cf5b39067 [id=c183d46e9c6e40d79c785d68a1f0dc1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.783269 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9f325dc993432fa07aaf099b76ca85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.787017 2268 ts_tablet_manager.cc:1916] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.787446 2276 ts_tablet_manager.cc:1916] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.789870 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 2d393766f3f54c9db6a1fc4108bb0345 (DEFAULT_TABLE table=default.Table_7a2d015cb9324e77afad1280b0e8081c [id=27814470bfe94b1897f6d89c13710f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.790171 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d393766f3f54c9db6a1fc4108bb0345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.764343 1176 tablet_bootstrap.cc:492] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.778258 748 tablet_service.cc:1511] Processing CreateTablet for tablet 7839b616fc854974b0dca43aead118f4 (DEFAULT_TABLE table=default.Table_eebd7fed9d9d4b4c923a82aa7d302199 [id=ea983cbbaeca4fdb9c466690a750a2cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.791785 1176 ts_tablet_manager.cc:1403] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.066s user 0.003s sys 0.000s
I20260505 07:54:18.792026 748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7839b616fc854974b0dca43aead118f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.792443 1176 raft_consensus.cc:359] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.792590 1176 raft_consensus.cc:385] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.792635 1176 raft_consensus.cc:740] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.792837 1176 consensus_queue.cc:260] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.778419 750 tablet_service.cc:1511] Processing CreateTablet for tablet 1d4858e05c744536a32c21e53ba2edf2 (DEFAULT_TABLE table=default.Table_153cefb63d3a4dfa894369fe70347fed [id=917e54c08fa34483b17d9d01efdfbefb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.795943 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d4858e05c744536a32c21e53ba2edf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.799626 1176 ts_tablet_manager.cc:1434] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
W20260505 07:54:18.800043 718 consensus_peers.cc:597] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f1feb0387bc4afead72346398dbe9e5. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:18.800504 1176 tablet_bootstrap.cc:492] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.801824 1176 tablet_bootstrap.cc:654] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.778443 756 tablet_service.cc:1511] Processing CreateTablet for tablet 11200db846bc40d490845671366a00f5 (DEFAULT_TABLE table=default.Table_8ed3074214464ce3ae998a9580562f57 [id=8311571b866f48d68b8a915588bb1934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.803926 756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11200db846bc40d490845671366a00f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.778761 745 tablet_service.cc:1511] Processing CreateTablet for tablet ee9f325dc993432fa07aaf099b76ca85 (DEFAULT_TABLE table=default.Table_7522f51dac38424194a5755cf5b39067 [id=c183d46e9c6e40d79c785d68a1f0dc1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.776139 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9c4fc18f6faa488882d9bffb30793cdd" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:18.807817 854 leader_election.cc:343] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.807945 854 leader_election.cc:304] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.807880 2447 ts_tablet_manager.cc:1929] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.808427 3144 raft_consensus.cc:2749] T 9c4fc18f6faa488882d9bffb30793cdd P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.808298 2447 log.cc:1199] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/e6a5f823db6143319d743d363a77ea47
I20260505 07:54:18.808773 749 tablet_service.cc:1511] Processing CreateTablet for tablet 2d393766f3f54c9db6a1fc4108bb0345 (DEFAULT_TABLE table=default.Table_7a2d015cb9324e77afad1280b0e8081c [id=27814470bfe94b1897f6d89c13710f6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.809049 749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d393766f3f54c9db6a1fc4108bb0345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.810070 1181 tablet_bootstrap.cc:492] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:18.810086 2447 ts_tablet_manager.cc:1950] T e6a5f823db6143319d743d363a77ea47 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:18.814555 899 tablet_service.cc:1511] Processing CreateTablet for tablet 095933526b83479ea20af9abbb1cce70 (DEFAULT_TABLE table=default.Table_921c2024ffef4029a90fe927e10bb705 [id=41a80c3f0be446478b8b35d21cab4638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.813401 3202 consensus_queue.cc:1048] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260505 07:54:18.814719 892 tablet_service.cc:1511] Processing CreateTablet for tablet d8f9af48a0e44983a033e06a84d86584 (DEFAULT_TABLE table=default.Table_19d6e7d3eebb4ebf9c35e22ba6a83ec3 [id=b884ea8753534dfa937e2fb0ad6874f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.815001 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 095933526b83479ea20af9abbb1cce70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.815671 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9f325dc993432fa07aaf099b76ca85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.815881 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f9af48a0e44983a033e06a84d86584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.813663 1028 tablet_service.cc:1511] Processing CreateTablet for tablet d8f9af48a0e44983a033e06a84d86584 (DEFAULT_TABLE table=default.Table_19d6e7d3eebb4ebf9c35e22ba6a83ec3 [id=b884ea8753534dfa937e2fb0ad6874f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.818181 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f9af48a0e44983a033e06a84d86584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.819674 3004 raft_consensus.cc:493] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:18.813854 1027 tablet_service.cc:1511] Processing CreateTablet for tablet d912673d3db446aea4b32c233349e387 (DEFAULT_TABLE table=default.Table_02f83ad68c3f4615b60d1fc64f984e5c [id=ad86e60549c245029484fe9733a2892e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.814036 1021 tablet_service.cc:1511] Processing CreateTablet for tablet 095933526b83479ea20af9abbb1cce70 (DEFAULT_TABLE table=default.Table_921c2024ffef4029a90fe927e10bb705 [id=41a80c3f0be446478b8b35d21cab4638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.823531 754 tablet_service.cc:1511] Processing CreateTablet for tablet d8f9af48a0e44983a033e06a84d86584 (DEFAULT_TABLE table=default.Table_19d6e7d3eebb4ebf9c35e22ba6a83ec3 [id=b884ea8753534dfa937e2fb0ad6874f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.823333 1027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d912673d3db446aea4b32c233349e387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.823858 744 tablet_service.cc:1511] Processing CreateTablet for tablet 095933526b83479ea20af9abbb1cce70 (DEFAULT_TABLE table=default.Table_921c2024ffef4029a90fe927e10bb705 [id=41a80c3f0be446478b8b35d21cab4638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.823871 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8f9af48a0e44983a033e06a84d86584. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.824652 1355 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_a31277b1c1ca4863976cb9726e83f13a" } new_table_name: "default.Table_a42a6f452cf742ffa4ff6d97be81ced6" modify_external_catalogs: true
I20260505 07:54:18.826022 1021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 095933526b83479ea20af9abbb1cce70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.826432 899 tablet_service.cc:1511] Processing CreateTablet for tablet 292e4aa7431c47698908a8e65b15fff6 (DEFAULT_TABLE table=default.Table_b89ce585388145c6932a0c87ba01a5ff [id=cb5870ca883f47dca42ee4ad72a4ee8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.827147 1357 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_f39ab9af40ff46588e5ea492e2e094c0" } new_table_name: "default.Table_6080bec05b6c4e419d157ea221b02c7c" modify_external_catalogs: true
I20260505 07:54:18.827674 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 292e4aa7431c47698908a8e65b15fff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.827849 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 095933526b83479ea20af9abbb1cce70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.829710 758 tablet_service.cc:1511] Processing CreateTablet for tablet d912673d3db446aea4b32c233349e387 (DEFAULT_TABLE table=default.Table_02f83ad68c3f4615b60d1fc64f984e5c [id=ad86e60549c245029484fe9733a2892e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.830060 758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d912673d3db446aea4b32c233349e387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.832633 1017 tablet_service.cc:1511] Processing CreateTablet for tablet 292e4aa7431c47698908a8e65b15fff6 (DEFAULT_TABLE table=default.Table_b89ce585388145c6932a0c87ba01a5ff [id=cb5870ca883f47dca42ee4ad72a4ee8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.832938 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 292e4aa7431c47698908a8e65b15fff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.836009 3144 raft_consensus.cc:493] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.836148 3144 raft_consensus.cc:515] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.836707 3144 leader_election.cc:290] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.838647 3202 consensus_queue.cc:1048] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.839705 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac029d9a3c540f58f393e79bf7be30e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.840049 885 tablet_service.cc:1511] Processing CreateTablet for tablet 2fce3c5c4c894d95ada9903810b53e03 (DEFAULT_TABLE table=default.Table_3cbf7ce6c7c145a39364c848ab951b5f [id=da91c64bb2ca4c5bbb56ccb943fa2a04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.840411 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fce3c5c4c894d95ada9903810b53e03. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:18.840577 854 leader_election.cc:343] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.841163 762 tablet_service.cc:1558] Processing DeleteTablet for tablet ff7ed926a3ff4e7993864de734ca3c50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.841223 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac029d9a3c540f58f393e79bf7be30e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:18.841364 1051 raft_consensus.cc:2468] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:18.841452 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 1852f7f4ff594639908b03730b68fe86 with delete_type TABLET_DATA_DELETED (replaced by tablet aab3850799254dda9716fd5bf9404159) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.841689 762 tablet_service.cc:1558] Processing DeleteTablet for tablet 60772a9697ba417b879dc6025eeb8eec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.841715 856 leader_election.cc:304] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.842046 762 tablet_service.cc:1511] Processing CreateTablet for tablet 292e4aa7431c47698908a8e65b15fff6 (DEFAULT_TABLE table=default.Table_b89ce585388145c6932a0c87ba01a5ff [id=cb5870ca883f47dca42ee4ad72a4ee8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.842722 3004 raft_consensus.cc:515] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:18.843402 758 tablet_service.cc:1558] Processing DeleteTablet for tablet 26f3544e74d941d286a2a0317b941ab6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:58 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.844918 743 tablet_service.cc:1558] Processing DeleteTablet for tablet 25072055f5534078adf5b5025614f951 with delete_type TABLET_DATA_DELETED (replaced by tablet 52e2db0240e04d6d8e948ebaedcc7d24) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.845074 761 tablet_service.cc:1558] Processing DeleteTablet for tablet f121fdd001ec4ea38ccd24a6afd9e51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:59 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.846141 3144 raft_consensus.cc:2804] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:18.846231 3144 raft_consensus.cc:493] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:18.846316 3144 raft_consensus.cc:3060] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.847069 762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 292e4aa7431c47698908a8e65b15fff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.848795 3144 raft_consensus.cc:515] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.849900 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac029d9a3c540f58f393e79bf7be30e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:18.850366 3144 leader_election.cc:290] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:18.850662 854 leader_election.cc:343] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.851295 1018 tablet_service.cc:1511] Processing CreateTablet for tablet 2fce3c5c4c894d95ada9903810b53e03 (DEFAULT_TABLE table=default.Table_3cbf7ce6c7c145a39364c848ab951b5f [id=da91c64bb2ca4c5bbb56ccb943fa2a04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.851630 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fce3c5c4c894d95ada9903810b53e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.851799 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac029d9a3c540f58f393e79bf7be30e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:18.851949 1051 raft_consensus.cc:3060] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:18.852735 896 tablet_service.cc:1511] Processing CreateTablet for tablet d73fd67f4b91479c8a53792f4f3779c2 (DEFAULT_TABLE table=default.Table_dfb010d3ab294c0aa6d95c417e8c6a4e [id=04bbd0b5ebc54dda81ea32f720f2cb9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.853097 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d73fd67f4b91479c8a53792f4f3779c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.854260 1051 raft_consensus.cc:2468] T 9ac029d9a3c540f58f393e79bf7be30e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:18.855669 3004 leader_election.cc:290] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:18.856163 856 leader_election.cc:304] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.856976 761 tablet_service.cc:1511] Processing CreateTablet for tablet 2fce3c5c4c894d95ada9903810b53e03 (DEFAULT_TABLE table=default.Table_3cbf7ce6c7c145a39364c848ab951b5f [id=da91c64bb2ca4c5bbb56ccb943fa2a04]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:18.857682 718 leader_election.cc:343] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 62d4bc3154704ba2bfde83bca10f812d
W20260505 07:54:18.859805 717 leader_election.cc:343] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 62d4bc3154704ba2bfde83bca10f812d
I20260505 07:54:18.859902 717 leader_election.cc:304] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:18.860150 3004 raft_consensus.cc:2749] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:18.860355 748 tablet_service.cc:1558] Processing DeleteTablet for tablet 06904a54b1c5463da31af0bc00fb606f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:53:56 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.860381 3144 raft_consensus.cc:2804] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:18.860541 3144 raft_consensus.cc:697] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:18.860872 761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fce3c5c4c894d95ada9903810b53e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.860841 3144 consensus_queue.cc:237] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.866273 1017 tablet_service.cc:1511] Processing CreateTablet for tablet d73fd67f4b91479c8a53792f4f3779c2 (DEFAULT_TABLE table=default.Table_dfb010d3ab294c0aa6d95c417e8c6a4e [id=04bbd0b5ebc54dda81ea32f720f2cb9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.866608 1017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d73fd67f4b91479c8a53792f4f3779c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.869192 744 tablet_service.cc:1511] Processing CreateTablet for tablet d73fd67f4b91479c8a53792f4f3779c2 (DEFAULT_TABLE table=default.Table_dfb010d3ab294c0aa6d95c417e8c6a4e [id=04bbd0b5ebc54dda81ea32f720f2cb9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.869709 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d73fd67f4b91479c8a53792f4f3779c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.869786 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 194) took 9226 ms. Trace:
I20260505 07:54:18.871335 1176 tablet_bootstrap.cc:492] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.871460 1176 ts_tablet_manager.cc:1403] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.071s user 0.002s sys 0.000s
I20260505 07:54:18.872151 1176 raft_consensus.cc:359] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.872308 1176 raft_consensus.cc:385] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.872375 1176 raft_consensus.cc:740] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.872623 1176 consensus_queue.cc:260] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.873081 1176 ts_tablet_manager.cc:1434] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:18.873351 1176 tablet_bootstrap.cc:492] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.874706 1176 tablet_bootstrap.cc:654] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.869879 2447 rpcz_store.cc:276] 0505 07:54:09.643070 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.643146 (+ 76us) service_pool.cc:224] Handling call
0505 07:54:18.719006 (+9075860us) ts_tablet_manager.cc:1151] Deleting tablet e6a5f823db6143319d743d363a77ea47
0505 07:54:18.719015 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.732308 (+ 13293us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.807725 (+ 75417us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.869767 (+ 62042us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9050612}
I20260505 07:54:18.876375 2447 tablet_replica.cc:333] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:18.876544 2447 raft_consensus.cc:2243] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.876667 2447 raft_consensus.cc:2272] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.878883 2447 ts_tablet_manager.cc:1916] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.879487 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet e6a5f823db6143319d743d363a77ea47 (table default.Table_61a6fc82edd5408f991f40a8d15394a5 [id=9930805fdf364f25aea11f859b50bc71]) successfully deleted
I20260505 07:54:18.873950 892 tablet_service.cc:1511] Processing CreateTablet for tablet d912673d3db446aea4b32c233349e387 (DEFAULT_TABLE table=default.Table_02f83ad68c3f4615b60d1fc64f984e5c [id=ad86e60549c245029484fe9733a2892e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.880110 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d912673d3db446aea4b32c233349e387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.880311 2276 ts_tablet_manager.cc:1929] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:18.880434 2276 log.cc:1199] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/ae829daad64545e687a9528e5c38b9ed
I20260505 07:54:18.810503 1181 ts_tablet_manager.cc:1403] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.106s user 0.001s sys 0.008s
I20260505 07:54:18.880740 2276 ts_tablet_manager.cc:1950] T ae829daad64545e687a9528e5c38b9ed P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:18.884155 1181 raft_consensus.cc:359] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.886186 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 195) took 8129 ms. Trace:
I20260505 07:54:18.886301 2276 rpcz_store.cc:276] 0505 07:54:10.756497 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.756556 (+ 59us) service_pool.cc:224] Handling call
0505 07:54:18.780167 (+8023611us) ts_tablet_manager.cc:1151] Deleting tablet ae829daad64545e687a9528e5c38b9ed
0505 07:54:18.780283 (+ 116us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.808921 (+ 28638us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.880227 (+ 71306us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.886149 (+ 5922us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8023480}
I20260505 07:54:18.886686 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet ae829daad64545e687a9528e5c38b9ed (table default.Table_06ea1b3a616b4c5aa573a1c654538879 [id=a648fb7ba63246c38e2a9f120d6d0396]) successfully deleted
I20260505 07:54:18.887833 1181 raft_consensus.cc:385] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.888619 1172 tablet_bootstrap.cc:492] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:18.888850 1172 ts_tablet_manager.cc:1403] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.152s user 0.002s sys 0.006s
I20260505 07:54:18.889125 1176 tablet_bootstrap.cc:492] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.889249 1176 ts_tablet_manager.cc:1403] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20260505 07:54:18.890797 1176 raft_consensus.cc:359] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.891075 754 tablet_service.cc:1511] Processing CreateTablet for tablet abadd3af175e466db81491b8a69a6d7d (DEFAULT_TABLE table=default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.891273 1176 raft_consensus.cc:385] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.891628 1176 raft_consensus.cc:740] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.891865 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abadd3af175e466db81491b8a69a6d7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.891819 885 tablet_service.cc:1511] Processing CreateTablet for tablet bef4f466262743df88c9d0f5f05f022d (DEFAULT_TABLE table=default.Table_e70b90b3e24449e3953b4a25068a50e5 [id=6c7e795451994a9c96e8918b4829bb26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.891863 1176 consensus_queue.cc:260] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.892336 896 tablet_service.cc:1511] Processing CreateTablet for tablet 70ec051ac96144169f87a11f90667771 (DEFAULT_TABLE table=default.Table_5dd32763402445e9855bb9fe9c15dbdf [id=73121d56c6d54c3ca2ae0bc2895bae36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.892273 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bef4f466262743df88c9d0f5f05f022d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.893524 1176 ts_tablet_manager.cc:1434] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:18.894688 1176 tablet_bootstrap.cc:492] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.895201 745 tablet_service.cc:1511] Processing CreateTablet for tablet bef4f466262743df88c9d0f5f05f022d (DEFAULT_TABLE table=default.Table_e70b90b3e24449e3953b4a25068a50e5 [id=6c7e795451994a9c96e8918b4829bb26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.895555 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bef4f466262743df88c9d0f5f05f022d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.896431 2276 tablet_replica.cc:333] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:18.896656 2276 raft_consensus.cc:2243] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:18.896986 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70ec051ac96144169f87a11f90667771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.897884 3004 consensus_queue.cc:1048] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.898146 1176 tablet_bootstrap.cc:654] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.902642 2447 ts_tablet_manager.cc:1929] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.902745 2447 log.cc:1199] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/c1a61c1d03d544c8aa01537ead97f83c
I20260505 07:54:18.903124 2447 ts_tablet_manager.cc:1950] T c1a61c1d03d544c8aa01537ead97f83c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:18.897011 2276 raft_consensus.cc:2272] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.904385 2276 ts_tablet_manager.cc:1916] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.911072 1176 tablet_bootstrap.cc:492] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:18.911185 1176 ts_tablet_manager.cc:1403] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260505 07:54:18.913618 1176 raft_consensus.cc:359] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.913774 1176 raft_consensus.cc:385] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.913825 1176 raft_consensus.cc:740] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.889997 1172 raft_consensus.cc:359] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.890705 1033 tablet_service.cc:1511] Processing CreateTablet for tablet abadd3af175e466db81491b8a69a6d7d (DEFAULT_TABLE table=default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.916294 1033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abadd3af175e466db81491b8a69a6d7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.916528 3004 consensus_queue.cc:1048] T 29a82ed2b87c487aac84194f2dcb0d6b P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.917249 1176 consensus_queue.cc:260] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.891202 1018 tablet_service.cc:1511] Processing CreateTablet for tablet bef4f466262743df88c9d0f5f05f022d (DEFAULT_TABLE table=default.Table_e70b90b3e24449e3953b4a25068a50e5 [id=6c7e795451994a9c96e8918b4829bb26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.892787 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 70ec051ac96144169f87a11f90667771 (DEFAULT_TABLE table=default.Table_5dd32763402445e9855bb9fe9c15dbdf [id=73121d56c6d54c3ca2ae0bc2895bae36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.918175 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70ec051ac96144169f87a11f90667771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.919297 1176 ts_tablet_manager.cc:1434] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260505 07:54:18.925546 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bef4f466262743df88c9d0f5f05f022d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.928350 1176 tablet_bootstrap.cc:492] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:18.929876 1176 tablet_bootstrap.cc:654] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:18.930862 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 195) took 9287 ms. Trace:
I20260505 07:54:18.930943 2447 rpcz_store.cc:276] 0505 07:54:09.643393 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.643453 (+ 60us) service_pool.cc:224] Handling call
0505 07:54:18.876337 (+9232884us) ts_tablet_manager.cc:1151] Deleting tablet c1a61c1d03d544c8aa01537ead97f83c
0505 07:54:18.876345 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.899057 (+ 22712us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.902596 (+ 3539us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.930844 (+ 28248us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9207903}
I20260505 07:54:18.931411 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet c1a61c1d03d544c8aa01537ead97f83c (table default.Table_575ab5b128ee4a1baebbe1f90baf005a [id=cdb574c7a4e14a5ea7f3c8b036a8c336]) successfully deleted
I20260505 07:54:18.933202 3004 consensus_queue.cc:1048] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.936607 745 tablet_service.cc:1558] Processing DeleteTablet for tablet 7219f9896fd94187ae714635077110af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:00 UTC) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:18.938822 2268 ts_tablet_manager.cc:1929] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.938963 2268 log.cc:1199] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/b5d113a872f9400c8f28d0239ca42377
I20260505 07:54:18.939157 2447 tablet_replica.cc:333] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:18.939447 2447 raft_consensus.cc:2243] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:18.939872 2447 pending_rounds.cc:70] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: Trying to abort 1 pending ops.
I20260505 07:54:18.939941 2447 pending_rounds.cc:77] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555516640333824 op_type: NO_OP noop_request { }
I20260505 07:54:18.940048 2447 raft_consensus.cc:2889] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:18.940145 2447 raft_consensus.cc:2272] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.940703 2447 ts_tablet_manager.cc:1916] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.939364 2268 ts_tablet_manager.cc:1950] T b5d113a872f9400c8f28d0239ca42377 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:18.948333 1181 raft_consensus.cc:740] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.948961 1181 consensus_queue.cc:260] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:18.949182 1354 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.105.1:36755 (request call id 149) took 1657 ms (client timeout 1000 ms). Trace:
I20260505 07:54:18.949484 1172 raft_consensus.cc:385] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:18.949751 1172 raft_consensus.cc:740] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:18.919060 899 tablet_service.cc:1511] Processing CreateTablet for tablet abadd3af175e466db81491b8a69a6d7d (DEFAULT_TABLE table=default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.927309 2276 ts_tablet_manager.cc:1929] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.950510 1172 consensus_queue.cc:260] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:18.950670 2276 log.cc:1199] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/09b46ff0101a4e529077b9e30492ace7
I20260505 07:54:18.951263 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abadd3af175e466db81491b8a69a6d7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.951318 2276 ts_tablet_manager.cc:1950] T 09b46ff0101a4e529077b9e30492ace7 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:18.953152 3254 consensus_queue.cc:1048] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.953568 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 286) took 6192 ms. Trace:
I20260505 07:54:18.956373 745 tablet_service.cc:1511] Processing CreateTablet for tablet 70ec051ac96144169f87a11f90667771 (DEFAULT_TABLE table=default.Table_5dd32763402445e9855bb9fe9c15dbdf [id=73121d56c6d54c3ca2ae0bc2895bae36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.956785 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70ec051ac96144169f87a11f90667771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.957075 1181 ts_tablet_manager.cc:1434] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.076s user 0.001s sys 0.001s
I20260505 07:54:18.957350 1181 tablet_bootstrap.cc:492] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:18.959914 892 tablet_service.cc:1511] Processing CreateTablet for tablet f6d59c9bfc3b4712907d13ea143adb17 (DEFAULT_TABLE table=default.Table_85003b58445a4d7fa3848473b7a93d08 [id=e47c1f5f60e647748d158c61c9e75f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.960515 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6d59c9bfc3b4712907d13ea143adb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.961038 1032 tablet_service.cc:1511] Processing CreateTablet for tablet 03a3d05438e14396aab2bb4e7348feeb (DEFAULT_TABLE table=default.Table_21368b74b6e04fb4a13b91d767f1c592 [id=6edd74bcfd354a2f939a50a433d9f6d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.961367 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a3d05438e14396aab2bb4e7348feeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.961617 744 tablet_service.cc:1511] Processing CreateTablet for tablet f6d59c9bfc3b4712907d13ea143adb17 (DEFAULT_TABLE table=default.Table_85003b58445a4d7fa3848473b7a93d08 [id=e47c1f5f60e647748d158c61c9e75f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.961985 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6d59c9bfc3b4712907d13ea143adb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.963809 754 tablet_service.cc:1511] Processing CreateTablet for tablet 03a3d05438e14396aab2bb4e7348feeb (DEFAULT_TABLE table=default.Table_21368b74b6e04fb4a13b91d767f1c592 [id=6edd74bcfd354a2f939a50a433d9f6d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.964154 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a3d05438e14396aab2bb4e7348feeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.965909 896 tablet_service.cc:1511] Processing CreateTablet for tablet 03a3d05438e14396aab2bb4e7348feeb (DEFAULT_TABLE table=default.Table_21368b74b6e04fb4a13b91d767f1c592 [id=6edd74bcfd354a2f939a50a433d9f6d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.966387 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a3d05438e14396aab2bb4e7348feeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.970522 2447 ts_tablet_manager.cc:1929] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:18.970633 2447 log.cc:1199] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/d77ff90ec2ff4b6eb1795d149459c23a
I20260505 07:54:18.971000 2447 ts_tablet_manager.cc:1950] T d77ff90ec2ff4b6eb1795d149459c23a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:18.971757 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 196) took 9328 ms. Trace:
I20260505 07:54:18.971844 2447 rpcz_store.cc:276] 0505 07:54:09.643715 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.643780 (+ 65us) service_pool.cc:224] Handling call
0505 07:54:18.931221 (+9287441us) ts_tablet_manager.cc:1151] Deleting tablet d77ff90ec2ff4b6eb1795d149459c23a
0505 07:54:18.939097 (+ 7876us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.955399 (+ 16302us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.970479 (+ 15080us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.971739 (+ 1260us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9262745}
W20260505 07:54:18.949316 1354 rpcz_store.cc:269] 0505 07:54:17.291896 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.318506 (+ 26610us) service_pool.cc:224] Handling call
0505 07:54:17.398480 (+ 79974us) spinlock_profiling.cc:243] Waited 68.5 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d1358 00007fd65a467ca6 00007fd65a4679d5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:18.949128 (+1550648us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:17.623731 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.623870 (+ 139us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.623877 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:17.623882 (+ 5us) tablet.cc:662] Decoding operations
0505 07:54:17.624056 (+ 174us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.624069 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.624074 (+ 5us) tablet.cc:685] Acquiring locks for 3 operations
0505 07:54:17.624174 (+ 100us) tablet.cc:701] Row locks acquired
0505 07:54:17.624176 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.624211 (+ 35us) write_op.cc:270] Start()
0505 07:54:17.624246 (+ 35us) write_op.cc:276] Timestamp: P: 1777967657624168 usec, L: 0
0505 07:54:17.624249 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.624476 (+ 227us) log.cc:844] Serialized 1275 byte log entry
0505 07:54:17.628044 (+ 3568us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.628346 (+ 302us) write_op.cc:301] APPLY: starting
0505 07:54:17.628364 (+ 18us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.628442 (+ 78us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.628479 (+ 37us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.628606 (+ 127us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.628651 (+ 45us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0505 07:54:17.628662 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:17.628743 (+ 81us) log.cc:844] Serialized 35 byte log entry
0505 07:54:17.628807 (+ 64us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.628829 (+ 22us) write_op.cc:454] Released schema lock
0505 07:54:17.628837 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.628960 (+ 123us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.628983 (+ 23us) write_op.cc:454] Released schema lock
Related trace 'op':
0505 07:54:17.648462 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.648632 (+ 170us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.648641 (+ 9us) write_op.cc:435] Acquired schema lock
0505 07:54:17.648645 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:17.648768 (+ 123us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.648786 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.648793 (+ 7us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:17.648945 (+ 152us) tablet.cc:701] Row locks acquired
0505 07:54:17.648949 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:17.648991 (+ 42us) write_op.cc:270] Start()
0505 07:54:17.649033 (+ 42us) write_op.cc:276] Timestamp: P: 1777967657648941 usec, L: 0
0505 07:54:17.649039 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.649252 (+ 213us) log.cc:844] Serialized 356 byte log entry
0505 07:54:17.823967 (+174715us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.824164 (+ 197us) write_op.cc:301] APPLY: starting
0505 07:54:17.824183 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.824226 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.824294 (+ 68us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.824360 (+ 66us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.824391 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:17.824398 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:17.824472 (+ 74us) log.cc:844] Serialized 23 byte log entry
0505 07:54:17.824546 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.824580 (+ 34us) write_op.cc:454] Released schema lock
0505 07:54:17.824586 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.824653 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.824656 (+ 3us) write_op.cc:454] Released schema lock
Related trace 'op':
0505 07:54:17.867475 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:17.867567 (+ 92us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:17.867574 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:17.867576 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:17.867683 (+ 107us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:17.867695 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:17.867700 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:17.867754 (+ 54us) tablet.cc:701] Row locks acquired
0505 07:54:17.867756 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:17.867787 (+ 31us) write_op.cc:270] Start()
0505 07:54:17.867815 (+ 28us) write_op.cc:276] Timestamp: P: 1777967657867720 usec, L: 0
0505 07:54:17.867818 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:17.867980 (+ 162us) log.cc:844] Serialized 356 byte log entry
0505 07:54:17.884211 (+ 16231us) op_driver.cc:464] REPLICATION: finished
0505 07:54:17.884325 (+ 114us) write_op.cc:301] APPLY: starting
0505 07:54:17.884338 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:17.884375 (+ 37us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:17.884406 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:17.884451 (+ 45us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:17.884481 (+ 30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:17.884487 (+ 6us) write_op.cc:312] APPLY: finished
0505 07:54:17.884567 (+ 80us) log.cc:844] Serialized 23 byte log entry
0505 07:54:17.884618 (+ 51us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.884632 (+ 14us) write_op.cc:454] Released schema lock
0505 07:54:17.884637 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:17.884692 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:17.884696 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21839,"reported_tablets":281,"spinlock_wait_cycles":157441920,"child_traces":[["op",{"apply.queue_time_us":245,"apply.run_cpu_time_us":577,"apply.run_wall_time_us":577,"num_ops":3,"prepare.queue_time_us":65,"prepare.run_cpu_time_us":871,"prepare.run_wall_time_us":871,"raft.queue_time_us":122,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":261,"replication_time_us":3770}],["op",{"apply.queue_time_us":91,"apply.run_cpu_time_us":490,"apply.run_wall_time_us":490,"num_ops":1,"prepare.queue_time_us":7494,"prepare.run_cpu_time_us":846,"prepare.run_wall_time_us":889,"replication_time_us":174896}],["op",{"apply.queue_time_us":66,"apply.run_cpu_time_us":362,"apply.run_wall_time_us":362,"num_ops":1,"prepare.queue_time_us":36021,"prepare.run_cpu_time_us":632,"prepare.run_wall_time_us":632,"raft.queue_time_us":170,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":191,"replication_time_us":16365}]]}
I20260505 07:54:18.972271 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet d77ff90ec2ff4b6eb1795d149459c23a (table default.Table_1ff5286907c6464c8addf47c70eef7ef [id=f608e93cbc64485297c6dc2609a6eb87]) successfully deleted
I20260505 07:54:18.973376 1354 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_af7b8d3a4e6843348198094736404edb" } new_table_name: "default.Table_018af517d4374faaa03e4eba33184d90" modify_external_catalogs: true
I20260505 07:54:18.955927 2268 rpcz_store.cc:276] 0505 07:54:12.761602 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.761671 (+ 69us) service_pool.cc:224] Handling call
0505 07:54:18.770837 (+6009166us) ts_tablet_manager.cc:1151] Deleting tablet b5d113a872f9400c8f28d0239ca42377
0505 07:54:18.770844 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.886487 (+115643us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.938771 (+ 52284us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.953552 (+ 14781us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6008991}
I20260505 07:54:18.974671 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet b5d113a872f9400c8f28d0239ca42377 (table default.Table_e9f1ff58ca6d41f19d05c458dc529941 [id=7579207045ec4220b966af9016a8f050]) successfully deleted
I20260505 07:54:18.977859 2268 tablet_replica.cc:333] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:18.978250 2268 raft_consensus.cc:2243] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.979766 2268 raft_consensus.cc:2272] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.980520 2268 ts_tablet_manager.cc:1916] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.982489 3145 raft_consensus.cc:493] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.982640 3145 raft_consensus.cc:515] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.983255 3145 leader_election.cc:290] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.983345 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 194) took 8227 ms. Trace:
I20260505 07:54:18.983466 2276 rpcz_store.cc:276] 0505 07:54:10.756094 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.756457 (+ 363us) service_pool.cc:224] Handling call
0505 07:54:18.896361 (+8139904us) ts_tablet_manager.cc:1151] Deleting tablet 09b46ff0101a4e529077b9e30492ace7
0505 07:54:18.896370 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.924405 (+ 28035us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.927235 (+ 2830us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:18.983303 (+ 56068us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31146,"tablet-delete.queue_time_us":8139509}
I20260505 07:54:18.984282 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 09b46ff0101a4e529077b9e30492ace7 (table default.Table_c12be1928760438fab7c135df431bc79 [id=d567651314004cf4b33a8dfc1ef30698]) successfully deleted
I20260505 07:54:18.984477 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09019fdf60374b11bf3427f21a55b5e0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.984714 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 197) took 8227 ms. Trace:
I20260505 07:54:18.984797 2276 rpcz_store.cc:276] 0505 07:54:10.757323 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.757366 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:18.984600 (+8227234us) ts_tablet_manager.cc:1151] Deleting tablet a9494c3b6a99487596a030856b371b68
0505 07:54:18.984610 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:18.984673 (+ 63us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8227113}
I20260505 07:54:18.979131 1028 tablet_service.cc:1511] Processing CreateTablet for tablet f6d59c9bfc3b4712907d13ea143adb17 (DEFAULT_TABLE table=default.Table_85003b58445a4d7fa3848473b7a93d08 [id=e47c1f5f60e647748d158c61c9e75f19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:18.983857 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09019fdf60374b11bf3427f21a55b5e0" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:18.985188 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a9494c3b6a99487596a030856b371b68 because the tablet was not found. No further retry: Not found: Tablet not found: a9494c3b6a99487596a030856b371b68
I20260505 07:54:18.985469 2276 tablet_replica.cc:333] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:18.985591 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6d59c9bfc3b4712907d13ea143adb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:18.985766 2276 raft_consensus.cc:2243] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:18.985934 2276 raft_consensus.cc:2272] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:18.986436 856 leader_election.cc:343] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:18.986701 854 leader_election.cc:343] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:18.986888 1172 ts_tablet_manager.cc:1434] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.098s user 0.001s sys 0.001s
I20260505 07:54:18.986846 854 leader_election.cc:304] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:18.988267 3145 raft_consensus.cc:2749] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:18.988945 3200 consensus_queue.cc:1048] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.991173 2447 tablet_replica.cc:333] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:18.991385 2447 raft_consensus.cc:2243] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:18.992044 2447 raft_consensus.cc:2272] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:18.992606 2447 ts_tablet_manager.cc:1916] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:18.995321 3145 raft_consensus.cc:493] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:18.995460 3145 raft_consensus.cc:515] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:18.996078 3145 leader_election.cc:290] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:18.996768 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e08cf03ea0b495a94c6566ea6c8caad" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:18.996970 782 raft_consensus.cc:2468] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:18.998591 854 leader_election.cc:304] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:18.998910 3202 consensus_queue.cc:1048] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:18.999441 1172 tablet_bootstrap.cc:492] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:18.999614 1317 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.60:35401, user_credentials={real_user=slave}} blocked reactor thread for 51794.3us
I20260505 07:54:19.001288 1172 tablet_bootstrap.cc:654] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.003809 3144 raft_consensus.cc:2804] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.003901 3144 raft_consensus.cc:493] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.003990 3144 raft_consensus.cc:3060] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.004488 3145 raft_consensus.cc:493] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.004655 3145 raft_consensus.cc:515] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.005249 3145 leader_election.cc:290] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:19.006141 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfadf8810f842698d96b6c13ffa1216" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.006770 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfadf8810f842698d96b6c13ffa1216" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:19.007090 854 leader_election.cc:343] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:19.008078 856 leader_election.cc:343] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.008229 856 leader_election.cc:304] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.008986 3145 raft_consensus.cc:2749] T bdfadf8810f842698d96b6c13ffa1216 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.010128 3144 raft_consensus.cc:515] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.010916 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e08cf03ea0b495a94c6566ea6c8caad" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.011060 782 raft_consensus.cc:3060] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.013556 782 raft_consensus.cc:2468] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:19.013921 854 leader_election.cc:304] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.014261 3145 raft_consensus.cc:2804] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.014360 3145 raft_consensus.cc:697] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:19.014580 3145 consensus_queue.cc:237] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.021607 3144 leader_election.cc:290] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.022076 3144 raft_consensus.cc:493] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.022238 3144 raft_consensus.cc:515] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.023500 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1d3829ede4b5582a022b64265df7e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:19.024047 854 leader_election.cc:343] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.026944 3144 leader_election.cc:290] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.032099 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e08cf03ea0b495a94c6566ea6c8caad" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:19.044806 856 leader_election.cc:343] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.044978 3199 raft_consensus.cc:493] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.045121 3199 raft_consensus.cc:515] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.045218 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1d3829ede4b5582a022b64265df7e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:19.045673 856 leader_election.cc:343] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.045732 3199 leader_election.cc:290] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.045804 856 leader_election.cc:304] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.046202 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e08cf03ea0b495a94c6566ea6c8caad" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:19.046646 856 leader_election.cc:343] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.046995 1181 tablet_bootstrap.cc:654] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.047019 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6546745e894545ff9c5f91bcc6e1a41e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.047343 1047 raft_consensus.cc:2468] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:19.048092 717 leader_election.cc:304] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.048543 3177 raft_consensus.cc:2804] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.048687 3177 raft_consensus.cc:493] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.048826 3177 raft_consensus.cc:3060] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.050022 3296 raft_consensus.cc:2749] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:19.052042 425 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 81141.6us
I20260505 07:54:19.052459 3199 raft_consensus.cc:493] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.052585 3199 raft_consensus.cc:515] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.053182 3199 leader_election.cc:290] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.057420 3199 raft_consensus.cc:493] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:19.057524 3199 raft_consensus.cc:515] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.057950 3199 leader_election.cc:290] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.058106 2447 ts_tablet_manager.cc:1929] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.058208 2447 log.cc:1199] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/3f1feb0387bc4afead72346398dbe9e5
I20260505 07:54:19.058539 2447 ts_tablet_manager.cc:1950] T 3f1feb0387bc4afead72346398dbe9e5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.058571 3199 raft_consensus.cc:493] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:19.058671 3199 raft_consensus.cc:515] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:19.059135 3199 leader_election.cc:290] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.059314 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 197) took 9415 ms. Trace:
I20260505 07:54:19.059387 2447 rpcz_store.cc:276] 0505 07:54:09.643863 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.644080 (+ 217us) service_pool.cc:224] Handling call
0505 07:54:18.991010 (+9346930us) ts_tablet_manager.cc:1151] Deleting tablet 3f1feb0387bc4afead72346398dbe9e5
0505 07:54:18.991116 (+ 106us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.038461 (+ 47345us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.058048 (+ 19587us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.059294 (+ 1246us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9322490}
I20260505 07:54:19.060878 2268 ts_tablet_manager.cc:1929] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.060961 2268 log.cc:1199] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/c4dd3ece5cb34ceb82feeb67e75da0b5
I20260505 07:54:19.061301 2268 ts_tablet_manager.cc:1950] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:19.063112 1181 tablet_bootstrap.cc:492] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.063216 1181 ts_tablet_manager.cc:1403] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.106s user 0.002s sys 0.001s
I20260505 07:54:19.063898 1181 raft_consensus.cc:359] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.064028 1181 raft_consensus.cc:385] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.064078 1181 raft_consensus.cc:740] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.064249 1181 consensus_queue.cc:260] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.064700 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225c56ac920b448c89c38d6fb3c1bdef" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:19.065169 717 leader_election.cc:343] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: feeafd614b3e4281a8e45b950e699b63
W20260505 07:54:19.065665 717 leader_election.cc:343] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 8a60c015136246a3b02f70254c0bb67f
I20260505 07:54:19.066044 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 03260a8507804e548486c81225de3ad9 (table default.Table_115e3886a51546409c5e744efd8b0462 [id=3c21c289d8f94c70b4ba7ef64dabc072]). Expected version 1 got 0
W20260505 07:54:19.066296 718 leader_election.cc:343] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: feeafd614b3e4281a8e45b950e699b63
I20260505 07:54:19.066661 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:19.066957 718 leader_election.cc:304] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.067209 3200 raft_consensus.cc:2749] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:19.067394 718 leader_election.cc:343] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 8a60c015136246a3b02f70254c0bb67f
I20260505 07:54:19.067520 718 leader_election.cc:304] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.067565 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 287) took 6255 ms. Trace:
I20260505 07:54:19.067799 3200 raft_consensus.cc:2749] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:19.067682 2268 rpcz_store.cc:276] 0505 07:54:12.812221 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.812274 (+ 53us) service_pool.cc:224] Handling call
0505 07:54:18.977816 (+6165542us) ts_tablet_manager.cc:1151] Deleting tablet c4dd3ece5cb34ceb82feeb67e75da0b5
0505 07:54:18.977825 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.048295 (+ 70470us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.060833 (+ 12538us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.067548 (+ 6715us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1486,"spinlock_wait_cycles":896,"tablet-delete.queue_time_us":6165373}
W20260505 07:54:19.068042 718 leader_election.cc:343] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 6546745e894545ff9c5f91bcc6e1a41e
I20260505 07:54:19.068114 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "225c56ac920b448c89c38d6fb3c1bdef" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.068413 907 raft_consensus.cc:2393] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:19.068452 2268 tablet_replica.cc:333] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:19.068794 1181 ts_tablet_manager.cc:1434] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:54:19.068843 718 leader_election.cc:304] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.068977 2268 raft_consensus.cc:2243] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.069105 3200 raft_consensus.cc:3060] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.069252 2268 raft_consensus.cc:2272] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.071539 3200 raft_consensus.cc:2749] T 225c56ac920b448c89c38d6fb3c1bdef P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.074784 3177 raft_consensus.cc:515] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:19.075990 718 leader_election.cc:343] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 6546745e894545ff9c5f91bcc6e1a41e
I20260505 07:54:19.068235 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet c4dd3ece5cb34ceb82feeb67e75da0b5 (table default.Table_d4c7087457154e89b20abe713355e422 [id=7099b9d0095f4bf880cff569eeeb5d32]) successfully deleted
W20260505 07:54:19.076298 3290 log.cc:927] Time spent T 0bd2e2514c72419390c1bb9bd13f1fb3 P d2799f5c6b4f4336b976ca9bbb458533: Append to log took a long time: real 0.084s user 0.000s sys 0.000s
I20260505 07:54:18.986443 2276 ts_tablet_manager.cc:1916] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.076725 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:19.077340 3177 leader_election.cc:290] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.077360 2268 ts_tablet_manager.cc:1916] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.078819 3298 raft_consensus.cc:493] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.079013 3298 raft_consensus.cc:515] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.079843 1181 tablet_bootstrap.cc:492] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.079903 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacf715cb07d4fe9b2c57caea594ab87" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:19.080367 856 leader_election.cc:343] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.081475 1181 tablet_bootstrap.cc:654] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.081812 1176 tablet_bootstrap.cc:492] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.081928 1176 ts_tablet_manager.cc:1403] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.154s user 0.013s sys 0.000s
I20260505 07:54:19.082422 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:19.082574 1176 raft_consensus.cc:359] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.082717 1176 raft_consensus.cc:385] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.082763 1176 raft_consensus.cc:740] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.083086 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:19.083093 1176 consensus_queue.cc:260] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.083556 1176 ts_tablet_manager.cc:1434] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:19.083827 1176 tablet_bootstrap.cc:492] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.083447 1351 catalog_manager.cc:5671] T 9cbfb2345a1a4b218c2dddbae6667e47 P d63a3e535f8e4dd4a88e447253405b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: HEALTHY } } }
I20260505 07:54:19.084967 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:19.085171 1176 tablet_bootstrap.cc:654] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.085335 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:19.086017 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet e243711164f04faabbde53b0a09d90b1 (table default.Table_a3cc6e739a71414f9f925aa759a07bd3 [id=5993b31e5350440f9715f1e416b90eb2]). Expected version 1 got 0
I20260505 07:54:19.086701 3298 leader_election.cc:290] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:19.087206 3261 raft_consensus.cc:493] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.087321 3261 raft_consensus.cc:515] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.087620 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:19.088223 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aacf715cb07d4fe9b2c57caea594ab87" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.088313 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6546745e894545ff9c5f91bcc6e1a41e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:19.088443 1051 raft_consensus.cc:3060] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:19.088717 854 leader_election.cc:343] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.088907 2447 tablet_replica.cc:333] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.088869 854 leader_election.cc:304] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.089105 2447 raft_consensus.cc:2243] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.087883 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 3f1feb0387bc4afead72346398dbe9e5 (table default.Table_dcf56067c2654eff8773e49d0ddf5427 [id=a9ef91bfcff640abb23140249a59fcf4]) successfully deleted
I20260505 07:54:19.089767 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:19.090072 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e243711164f04faabbde53b0a09d90b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.090257 775 raft_consensus.cc:2468] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.090268 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:19.090466 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:19.090699 988 leader_election.cc:304] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.090706 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:19.090888 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 490483562d8f4c2daa7eaa333a9f3a0c (table default.Table_3acf7c3eb13d473a9d952098c7524753 [id=fe7ee6e8118840e8a9f70a007d80c2b3]). Expected version 1 got 0
I20260505 07:54:19.090911 1051 raft_consensus.cc:2468] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:19.090973 3173 raft_consensus.cc:2804] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.091055 3173 raft_consensus.cc:493] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.091131 3173 raft_consensus.cc:3060] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.091171 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:19.091252 3261 leader_election.cc:290] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.091317 717 leader_election.cc:304] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:19.091712 3177 raft_consensus.cc:2804] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.091920 3177 raft_consensus.cc:697] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:19.092039 3145 raft_consensus.cc:2749] T aacf715cb07d4fe9b2c57caea594ab87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.092136 3177 consensus_queue.cc:237] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.093276 2447 pending_rounds.cc:70] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: Trying to abort 1 pending ops.
I20260505 07:54:19.093329 2447 pending_rounds.cc:77] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555461670260736 op_type: NO_OP noop_request { }
I20260505 07:54:19.093407 2447 raft_consensus.cc:2889] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:19.093495 2447 raft_consensus.cc:2272] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.093617 3173 raft_consensus.cc:515] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.094274 3173 leader_election.cc:290] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.094751 2447 ts_tablet_manager.cc:1916] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.094868 2268 ts_tablet_manager.cc:1929] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:19.094951 2268 log.cc:1199] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/571257bbb7e94315b60d84d6aa4917b1
I20260505 07:54:19.095274 2268 ts_tablet_manager.cc:1950] T 571257bbb7e94315b60d84d6aa4917b1 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:19.095940 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 288) took 5668 ms. Trace:
I20260505 07:54:19.096153 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e243711164f04faabbde53b0a09d90b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.096378 912 raft_consensus.cc:2468] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.096885 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e243711164f04faabbde53b0a09d90b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.097082 775 raft_consensus.cc:3060] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.098135 3298 raft_consensus.cc:493] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.098254 3298 raft_consensus.cc:515] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.098868 3298 leader_election.cc:290] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:19.099279 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12d2d2b866348d781209fa787da7d7e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.099854 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 025c94c839484adba26945b62a217f5c (table default.Table_96e345e18b864cce8826bf32cac7ef52 [id=327b63fb4f744980a47bdce7db242629]). Expected version 1 got 0
I20260505 07:54:19.100143 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12d2d2b866348d781209fa787da7d7e" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:19.100200 854 leader_election.cc:343] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.100906 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:19.101197 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:19.101225 775 raft_consensus.cc:2468] T e243711164f04faabbde53b0a09d90b1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:19.101344 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_1c26ba8978f749a38cfdb26ff048283c [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 2 got 0
I20260505 07:54:19.101500 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_90c98f62112f465294c60c56f6835a93 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 2 got 0
I20260505 07:54:19.101637 988 leader_election.cc:304] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.101912 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
W20260505 07:54:19.102062 856 leader_election.cc:343] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.102173 856 leader_election.cc:304] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.102378 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:19.102620 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:19.102774 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 90612f0afb4949f491e625c61e40d247 (table default.Table_61a6fc82edd5408f991f40a8d15394a5 [id=9930805fdf364f25aea11f859b50bc71]). Expected version 1 got 0
I20260505 07:54:19.102945 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:19.103305 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 823cd847ae2b496ca7e99998297ba2fe (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:19.103366 3202 raft_consensus.cc:2804] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.103487 3202 raft_consensus.cc:697] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:19.103524 3203 raft_consensus.cc:493] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.103621 3203 raft_consensus.cc:515] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.103765 3202 consensus_queue.cc:237] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.104249 3203 leader_election.cc:290] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.105242 1181 tablet_bootstrap.cc:492] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.105355 1181 ts_tablet_manager.cc:1403] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.026s user 0.002s sys 0.000s
I20260505 07:54:19.105553 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1506cd31fe364e328867bbf70e36845e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:19.106091 988 leader_election.cc:343] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.106144 1181 raft_consensus.cc:359] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.107949 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1506cd31fe364e328867bbf70e36845e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.108065 1181 raft_consensus.cc:385] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.108134 1181 raft_consensus.cc:740] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.108145 912 raft_consensus.cc:2468] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.108327 1181 consensus_queue.cc:260] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.108673 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e243711164f04faabbde53b0a09d90b1" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.108831 907 raft_consensus.cc:3060] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.109366 1181 ts_tablet_manager.cc:1434] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:19.109606 1181 tablet_bootstrap.cc:492] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.112123 907 raft_consensus.cc:2468] T e243711164f04faabbde53b0a09d90b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:19.112604 1181 tablet_bootstrap.cc:654] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.112970 991 leader_election.cc:304] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.113200 3203 raft_consensus.cc:2804] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.113271 3203 raft_consensus.cc:493] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.113349 3203 raft_consensus.cc:3060] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.096010 2268 rpcz_store.cc:276] 0505 07:54:13.428034 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.428790 (+ 756us) service_pool.cc:224] Handling call
0505 07:54:19.068001 (+5639211us) ts_tablet_manager.cc:1151] Deleting tablet 571257bbb7e94315b60d84d6aa4917b1
0505 07:54:19.068412 (+ 411us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.092115 (+ 23703us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.094826 (+ 2711us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.095923 (+ 1097us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":343,"tablet-delete.queue_time_us":5638982}
I20260505 07:54:19.116220 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 289) took 5615 ms. Trace:
I20260505 07:54:19.116284 2268 rpcz_store.cc:276] 0505 07:54:13.500454 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.500542 (+ 88us) service_pool.cc:224] Handling call
0505 07:54:19.116120 (+5615578us) ts_tablet_manager.cc:1151] Deleting tablet 4466d0ed977e4803b3b484155095c271
0505 07:54:19.116128 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.116207 (+ 79us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5615329}
I20260505 07:54:19.116526 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 290) took 5615 ms. Trace:
I20260505 07:54:19.116576 2268 rpcz_store.cc:276] 0505 07:54:13.501066 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.501124 (+ 58us) service_pool.cc:224] Handling call
0505 07:54:19.116455 (+5615331us) ts_tablet_manager.cc:1151] Deleting tablet dfbe4dff192741099be5ab10a96adeb7
0505 07:54:19.116462 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.116516 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5615156}
I20260505 07:54:19.116808 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 295) took 4568 ms. Trace:
I20260505 07:54:19.116863 2268 rpcz_store.cc:276] 0505 07:54:14.548070 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.548132 (+ 62us) service_pool.cc:224] Handling call
0505 07:54:19.116721 (+4568589us) ts_tablet_manager.cc:1151] Deleting tablet 9f541eff399641dd81a0771186805493
0505 07:54:19.116727 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.116795 (+ 68us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4568417}
I20260505 07:54:19.117106 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 304) took 4451 ms. Trace:
I20260505 07:54:19.117173 2268 rpcz_store.cc:276] 0505 07:54:14.666060 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.666102 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:19.117035 (+4450933us) ts_tablet_manager.cc:1151] Deleting tablet 872362aecf6a44b59e586b1fa7637c8b
0505 07:54:19.117044 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.117093 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4450780}
I20260505 07:54:19.117424 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 305) took 4450 ms. Trace:
I20260505 07:54:19.117484 2268 rpcz_store.cc:276] 0505 07:54:14.666746 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.666790 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:19.117355 (+4450565us) ts_tablet_manager.cc:1151] Deleting tablet 6ca2852cee84476dbe5e5072a43087a1
0505 07:54:19.117364 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.117411 (+ 47us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4450423}
I20260505 07:54:19.117676 2268 tablet_replica.cc:333] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:19.117782 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 9926e569032144ec9d3148f4561333ab (table default.Table_3f9462e44d2b4d138eb0510c4a571080 [id=ed71bc1104b5423caedd9268977cc018]). Expected version 1 got 0
I20260505 07:54:19.118196 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:19.119084 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 89d3e08134704859abc44257f484b8cd (table default.Table_e9f1ff58ca6d41f19d05c458dc529941 [id=7579207045ec4220b966af9016a8f050]). Expected version 1 got 0
I20260505 07:54:19.119663 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 27ab0ebc13f349f38cf86b9643700ae1 (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:19.120010 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet e38f248556bd4e49b06da72a1cbd7d8f (table default.Table_5c2b133923064d5eb9eabc21fb62f098 [id=12443115e63e4428a9cb798980fcbf5b]). Expected version 1 got 0
I20260505 07:54:19.123227 3203 raft_consensus.cc:515] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.123818 3203 leader_election.cc:290] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.120231 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 571257bbb7e94315b60d84d6aa4917b1 (table default.Table_4ab0ac01b8bf49ef96683dca41dfadd4 [id=6cb8bc8043474bad9f4278b7eef7c71a]) successfully deleted
W20260505 07:54:19.124415 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 4466d0ed977e4803b3b484155095c271 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4466d0ed977e4803b3b484155095c271 already in progress: creating tablet
W20260505 07:54:19.124665 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet dfbe4dff192741099be5ab10a96adeb7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dfbe4dff192741099be5ab10a96adeb7 already in progress: creating tablet
I20260505 07:54:19.124708 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1506cd31fe364e328867bbf70e36845e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:19.124855 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 9f541eff399641dd81a0771186805493 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9f541eff399641dd81a0771186805493 already in progress: creating tablet
W20260505 07:54:19.125016 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 872362aecf6a44b59e586b1fa7637c8b because the tablet was not found. No further retry: Not found: Tablet not found: 872362aecf6a44b59e586b1fa7637c8b
W20260505 07:54:19.125185 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 6ca2852cee84476dbe5e5072a43087a1 because the tablet was not found. No further retry: Not found: Tablet not found: 6ca2852cee84476dbe5e5072a43087a1
I20260505 07:54:19.125959 1181 tablet_bootstrap.cc:492] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.126066 1181 ts_tablet_manager.cc:1403] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.003s
W20260505 07:54:19.126713 988 leader_election.cc:343] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.126694 1181 raft_consensus.cc:359] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.126960 1181 raft_consensus.cc:385] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.126902 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1506cd31fe364e328867bbf70e36845e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.127003 1181 raft_consensus.cc:740] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.127076 912 raft_consensus.cc:3060] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.127179 1181 consensus_queue.cc:260] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.127667 2447 ts_tablet_manager.cc:1929] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.127763 2447 log.cc:1199] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/4a79578757824bfc9df129e3475e7af3
I20260505 07:54:19.128108 2447 ts_tablet_manager.cc:1950] T 4a79578757824bfc9df129e3475e7af3 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.128508 1181 ts_tablet_manager.cc:1434] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:19.128878 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 198) took 9484 ms. Trace:
I20260505 07:54:19.128952 2447 rpcz_store.cc:276] 0505 07:54:09.644348 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.644410 (+ 62us) service_pool.cc:224] Handling call
0505 07:54:19.088854 (+9444444us) ts_tablet_manager.cc:1151] Deleting tablet 4a79578757824bfc9df129e3475e7af3
0505 07:54:19.088866 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.113445 (+ 24579us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.127586 (+ 14141us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.128860 (+ 1274us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9420283}
I20260505 07:54:19.131803 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 4a79578757824bfc9df129e3475e7af3 (table default.Table_b31d40ee188f46d0bdd481aa88730544 [id=c6653c0d02c64683870dc0e9551fcaf5]) successfully deleted
I20260505 07:54:19.132510 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:19.132704 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_65d14038399640c284b8382aa80e344f [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 2 got 0
I20260505 07:54:19.132961 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:19.133023 3133 consensus_queue.cc:1048] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.133144 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1506cd31fe364e328867bbf70e36845e (table default.Table_eee3bd15017341fca226f671948e94e9 [id=c7cad0401e40485e9ed03160b9bed704]). Expected version 1 got 0
I20260505 07:54:19.133682 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:19.134184 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:19.134337 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1bd5951bfc3346c2a6aad8f33357394a (table default.Table_4e651b0d817f408e8e36b0399a57a04f [id=d85dae5f1f424012929e0d90ef8b2d22]). Expected version 1 got 0
I20260505 07:54:19.134565 3144 raft_consensus.cc:2749] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.134690 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:19.135502 2268 raft_consensus.cc:2243] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.135680 2268 raft_consensus.cc:2272] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.136178 2268 ts_tablet_manager.cc:1916] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.138586 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:19.139123 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:19.139719 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:19.139953 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:19.140178 1351 catalog_manager.cc:5763] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:19.141790 2268 ts_tablet_manager.cc:1929] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.141867 2268 log.cc:1199] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/df1b62a30b2842b897a04c793bb5e7ce
I20260505 07:54:19.142176 2268 ts_tablet_manager.cc:1950] T df1b62a30b2842b897a04c793bb5e7ce P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:19.142802 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 306) took 4475 ms. Trace:
I20260505 07:54:19.142894 2268 rpcz_store.cc:276] 0505 07:54:14.667084 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.667139 (+ 55us) service_pool.cc:224] Handling call
0505 07:54:19.117643 (+4450504us) ts_tablet_manager.cc:1151] Deleting tablet df1b62a30b2842b897a04c793bb5e7ce
0505 07:54:19.117649 (+ 6us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.139039 (+ 21390us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.141744 (+ 2705us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.142778 (+ 1034us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17644,"tablet-delete.queue_time_us":4450380}
I20260505 07:54:19.143416 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet df1b62a30b2842b897a04c793bb5e7ce (table default.Table_6bd554849539457bb38dbbef32deb609 [id=269f2d2935a644518501cb0700a7cff9]) successfully deleted
I20260505 07:54:19.143863 1181 tablet_bootstrap.cc:492] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.145349 1181 tablet_bootstrap.cc:654] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.148518 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 307) took 4481 ms. Trace:
I20260505 07:54:19.148590 2268 rpcz_store.cc:276] 0505 07:54:14.667415 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.667451 (+ 36us) service_pool.cc:224] Handling call
0505 07:54:19.148440 (+4480989us) ts_tablet_manager.cc:1151] Deleting tablet 297537a92684425d8508f7b3b4c9d379
0505 07:54:19.148448 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.148503 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4480855}
I20260505 07:54:19.149411 3133 consensus_queue.cc:1048] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260505 07:54:19.149482 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 297537a92684425d8508f7b3b4c9d379 because the tablet was not found. No further retry: Not found: Tablet not found: 297537a92684425d8508f7b3b4c9d379
I20260505 07:54:19.149767 2268 tablet_replica.cc:333] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:19.149964 2268 raft_consensus.cc:2243] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.151698 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 199) took 9507 ms. Trace:
I20260505 07:54:19.151780 2447 rpcz_store.cc:276] 0505 07:54:09.644517 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.644581 (+ 64us) service_pool.cc:224] Handling call
0505 07:54:19.148149 (+9503568us) ts_tablet_manager.cc:1151] Deleting tablet 4662d4a3d88d4c2883a35048d6e11eb9
0505 07:54:19.148162 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.151678 (+ 3516us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9479516}
I20260505 07:54:19.152053 2447 tablet_replica.cc:333] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.152256 2447 raft_consensus.cc:2243] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:19.152411 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 4662d4a3d88d4c2883a35048d6e11eb9 because the tablet was not found. No further retry: Not found: Tablet not found: 4662d4a3d88d4c2883a35048d6e11eb9
I20260505 07:54:19.152406 2447 raft_consensus.cc:2272] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.156749 2447 ts_tablet_manager.cc:1916] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.161924 912 raft_consensus.cc:2468] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:19.162328 991 leader_election.cc:304] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.162850 3261 raft_consensus.cc:2804] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.162992 3261 raft_consensus.cc:697] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:19.163264 3261 consensus_queue.cc:237] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.163527 3202 consensus_queue.cc:1048] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.174043 1181 tablet_bootstrap.cc:492] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.174214 1181 ts_tablet_manager.cc:1403] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.030s user 0.001s sys 0.006s
I20260505 07:54:19.175058 1181 raft_consensus.cc:359] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.175515 1172 tablet_bootstrap.cc:492] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.175714 1172 ts_tablet_manager.cc:1403] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.176s user 0.005s sys 0.000s
I20260505 07:54:19.176437 1172 raft_consensus.cc:359] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.176616 1172 raft_consensus.cc:385] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.176685 1172 raft_consensus.cc:740] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.176911 1172 consensus_queue.cc:260] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.177451 1172 ts_tablet_manager.cc:1434] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:19.177750 1172 tablet_bootstrap.cc:492] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.179405 1172 tablet_bootstrap.cc:654] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.185068 3199 consensus_queue.cc:1048] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.189857 1181 raft_consensus.cc:385] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.189996 1181 raft_consensus.cc:740] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.189081 2268 raft_consensus.cc:2272] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.190552 3200 raft_consensus.cc:493] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.190682 3200 raft_consensus.cc:515] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.191510 3200 leader_election.cc:290] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.192476 3004 consensus_queue.cc:1048] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.193364 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235a3ccef904913bbd67c403aac3a87" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.193542 921 raft_consensus.cc:2468] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:19.190783 1181 consensus_queue.cc:260] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.197816 718 leader_election.cc:304] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.198051 3199 raft_consensus.cc:2804] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.198119 3199 raft_consensus.cc:493] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.198184 3199 raft_consensus.cc:3060] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.197258 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235a3ccef904913bbd67c403aac3a87" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.199302 1051 raft_consensus.cc:2468] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:19.199801 2268 ts_tablet_manager.cc:1916] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.200922 3199 raft_consensus.cc:515] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.201488 3199 leader_election.cc:290] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.202296 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235a3ccef904913bbd67c403aac3a87" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:19.202661 1051 raft_consensus.cc:3060] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.204041 921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235a3ccef904913bbd67c403aac3a87" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.204216 921 raft_consensus.cc:3060] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.206729 921 raft_consensus.cc:2468] T 7235a3ccef904913bbd67c403aac3a87 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:19.207854 718 leader_election.cc:304] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.208046 2447 ts_tablet_manager.cc:1929] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.208101 3004 raft_consensus.cc:2804] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.208712 3004 raft_consensus.cc:697] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:19.208948 3004 consensus_queue.cc:237] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.212591 1051 raft_consensus.cc:2468] T 7235a3ccef904913bbd67c403aac3a87 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:19.200264 1181 ts_tablet_manager.cc:1434] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.026s user 0.000s sys 0.001s
I20260505 07:54:19.220072 2447 log.cc:1199] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/8a60c015136246a3b02f70254c0bb67f
I20260505 07:54:19.220561 1181 tablet_bootstrap.cc:492] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.221166 2447 ts_tablet_manager.cc:1950] T 8a60c015136246a3b02f70254c0bb67f P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.222546 2276 ts_tablet_manager.cc:1929] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.222682 2276 log.cc:1199] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/4f3e076892234c1494b1a5c13cbfd2e2
I20260505 07:54:19.222873 1181 tablet_bootstrap.cc:654] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.223222 2276 ts_tablet_manager.cc:1950] T 4f3e076892234c1494b1a5c13cbfd2e2 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:19.224114 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 196) took 8467 ms. Trace:
I20260505 07:54:19.224233 2276 rpcz_store.cc:276] 0505 07:54:10.756970 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.757190 (+ 220us) service_pool.cc:224] Handling call
0505 07:54:18.985399 (+8228209us) ts_tablet_manager.cc:1151] Deleting tablet 4f3e076892234c1494b1a5c13cbfd2e2
0505 07:54:18.985408 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.080985 (+ 95577us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.222468 (+141483us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.224076 (+ 1608us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8227821}
I20260505 07:54:19.224859 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 4f3e076892234c1494b1a5c13cbfd2e2 (table default.Table_fe563815e7df4183a9bfe81907c3c33c [id=fa1b1c24909247538d57d1a37ea9513b]) successfully deleted
I20260505 07:54:19.225055 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 200) took 9580 ms. Trace:
I20260505 07:54:19.225131 2447 rpcz_store.cc:276] 0505 07:54:09.644822 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.644878 (+ 56us) service_pool.cc:224] Handling call
0505 07:54:19.152006 (+9507128us) ts_tablet_manager.cc:1151] Deleting tablet 8a60c015136246a3b02f70254c0bb67f
0505 07:54:19.152015 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.175853 (+ 23838us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.207986 (+ 32133us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.225037 (+ 17051us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9483329}
I20260505 07:54:19.225620 2276 tablet_replica.cc:333] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:19.225880 3261 raft_consensus.cc:493] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.226079 3261 raft_consensus.cc:515] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.227125 3261 leader_election.cc:290] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.227423 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e08cf03ea0b495a94c6566ea6c8caad" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.228365 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e08cf03ea0b495a94c6566ea6c8caad" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.231940 775 raft_consensus.cc:2393] T 0e08cf03ea0b495a94c6566ea6c8caad P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:19.232368 988 leader_election.cc:304] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.232486 2447 tablet_replica.cc:333] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.232425 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 8a60c015136246a3b02f70254c0bb67f (table default.Table_d9857ceb9b4d48dab8120e696f98e0e5 [id=65a99c8ba7b14ed0aec12382a2c8095c]) successfully deleted
I20260505 07:54:19.232877 3261 raft_consensus.cc:3060] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.234794 2268 ts_tablet_manager.cc:1929] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.234889 2268 log.cc:1199] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/080b2984fcf2478a8def6540da63554d
I20260505 07:54:19.235224 2268 ts_tablet_manager.cc:1950] T 080b2984fcf2478a8def6540da63554d P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:19.235641 3261 raft_consensus.cc:2749] T 0e08cf03ea0b495a94c6566ea6c8caad P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.235929 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 308) took 4568 ms. Trace:
I20260505 07:54:19.236066 2447 raft_consensus.cc:2243] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.236039 2268 rpcz_store.cc:276] 0505 07:54:14.667755 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.667907 (+ 152us) service_pool.cc:224] Handling call
0505 07:54:19.149713 (+4481806us) ts_tablet_manager.cc:1151] Deleting tablet 080b2984fcf2478a8def6540da63554d
0505 07:54:19.149723 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.230954 (+ 81231us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.234743 (+ 3789us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.235894 (+ 1151us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4481675}
I20260505 07:54:19.236507 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 080b2984fcf2478a8def6540da63554d (table default.Table_575ab5b128ee4a1baebbe1f90baf005a [id=cdb574c7a4e14a5ea7f3c8b036a8c336]) successfully deleted
I20260505 07:54:19.237167 2447 raft_consensus.cc:2272] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.238565 2447 ts_tablet_manager.cc:1916] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:19.238852 1315 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_65d14038399640c284b8382aa80e344f [id=fc9aba6852ee42cc83697f86e0e69c5e], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 68e608ac01ef4a88906e585bd7ee25ac consensus state has no leader
I20260505 07:54:19.239676 2268 tablet_replica.cc:333] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:19.240068 2268 raft_consensus.cc:2243] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.241050 1181 tablet_bootstrap.cc:492] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.241214 1181 ts_tablet_manager.cc:1403] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.005s
I20260505 07:54:19.241878 1181 raft_consensus.cc:359] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.242072 1181 raft_consensus.cc:385] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.242154 1181 raft_consensus.cc:740] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.242376 1181 consensus_queue.cc:260] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.242833 1181 ts_tablet_manager.cc:1434] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:19.243945 3199 raft_consensus.cc:493] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:19.244077 3199 raft_consensus.cc:515] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.244788 3199 leader_election.cc:290] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:19.245419 717 leader_election.cc:343] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: bc9ae6404dd64bd8bf4898768e94e4f1
W20260505 07:54:19.245733 718 leader_election.cc:343] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: bc9ae6404dd64bd8bf4898768e94e4f1
I20260505 07:54:19.245848 718 leader_election.cc:304] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.246223 3199 raft_consensus.cc:2749] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:19.253679 1172 tablet_bootstrap.cc:492] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.253876 1172 ts_tablet_manager.cc:1403] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.076s user 0.005s sys 0.004s
I20260505 07:54:19.254567 1172 raft_consensus.cc:359] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.254572 3173 consensus_queue.cc:1048] T 0e9a98243da14d43bc649bb19aed4f0c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.254815 1172 raft_consensus.cc:385] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.254925 1172 raft_consensus.cc:740] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.255191 1172 consensus_queue.cc:260] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.255734 1181 tablet_bootstrap.cc:492] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.262140 1181 tablet_bootstrap.cc:654] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.268122 3004 raft_consensus.cc:493] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:19.268323 3004 raft_consensus.cc:515] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
W20260505 07:54:19.269671 717 leader_election.cc:343] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 7001482d90364dbdb8b9c5bfb98e1cd4
I20260505 07:54:19.270757 1176 tablet_bootstrap.cc:492] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.269436 3004 leader_election.cc:290] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.270900 1176 ts_tablet_manager.cc:1403] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.187s user 0.020s sys 0.024s
I20260505 07:54:19.271508 2276 raft_consensus.cc:2243] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.271718 2276 raft_consensus.cc:2272] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:19.271960 718 leader_election.cc:343] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 7001482d90364dbdb8b9c5bfb98e1cd4
I20260505 07:54:19.272080 718 leader_election.cc:304] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.272188 1176 raft_consensus.cc:359] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.272370 3199 raft_consensus.cc:2749] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:19.272513 1176 raft_consensus.cc:385] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.273147 1176 raft_consensus.cc:740] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.273687 1176 consensus_queue.cc:260] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.274330 1176 ts_tablet_manager.cc:1434] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:19.274717 1176 tablet_bootstrap.cc:492] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.275508 2276 ts_tablet_manager.cc:1916] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:19.276202 717 consensus_peers.cc:597] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4dd3ece5cb34ceb82feeb67e75da0b5. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:19.277242 1172 ts_tablet_manager.cc:1434] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.023s user 0.001s sys 0.001s
I20260505 07:54:19.280612 1176 tablet_bootstrap.cc:654] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.281826 1181 tablet_bootstrap.cc:492] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.282117 1181 ts_tablet_manager.cc:1403] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.026s user 0.004s sys 0.009s
I20260505 07:54:19.282744 1181 raft_consensus.cc:359] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.282948 1181 raft_consensus.cc:385] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.283028 1181 raft_consensus.cc:740] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.283250 1181 consensus_queue.cc:260] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.286377 1181 ts_tablet_manager.cc:1434] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20260505 07:54:19.286652 1181 tablet_bootstrap.cc:492] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.288511 1181 tablet_bootstrap.cc:654] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.288961 3298 raft_consensus.cc:493] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.289222 3298 raft_consensus.cc:515] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.289866 3298 leader_election.cc:290] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.290517 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49d7b9f5b6f74f8ea070f8ea7e545df7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.290680 1051 raft_consensus.cc:2468] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:19.291204 856 leader_election.cc:304] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:19.291522 3298 raft_consensus.cc:2804] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.291616 3298 raft_consensus.cc:493] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.291684 3298 raft_consensus.cc:3060] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.292975 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49d7b9f5b6f74f8ea070f8ea7e545df7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.293198 782 raft_consensus.cc:2468] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:19.293834 2268 raft_consensus.cc:2272] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.295209 1172 tablet_bootstrap.cc:492] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.297749 2268 ts_tablet_manager.cc:1916] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.299055 1176 tablet_bootstrap.cc:492] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.299172 1176 ts_tablet_manager.cc:1403] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.001s
I20260505 07:54:19.299681 1176 raft_consensus.cc:359] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.299847 1176 raft_consensus.cc:385] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.299909 1176 raft_consensus.cc:740] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.300109 1176 consensus_queue.cc:260] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.300527 1176 ts_tablet_manager.cc:1434] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260505 07:54:19.301489 3298 raft_consensus.cc:515] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.306365 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49d7b9f5b6f74f8ea070f8ea7e545df7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:19.306501 1051 raft_consensus.cc:3060] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.308810 1051 raft_consensus.cc:2468] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:19.309242 856 leader_election.cc:304] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:19.309635 3296 raft_consensus.cc:2804] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.309744 3296 raft_consensus.cc:697] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:19.309975 1176 tablet_bootstrap.cc:492] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.309974 3296 consensus_queue.cc:237] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.311491 3298 leader_election.cc:290] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.311537 1176 tablet_bootstrap.cc:654] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.298439 1181 tablet_bootstrap.cc:492] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.312799 2447 ts_tablet_manager.cc:1929] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.312927 1181 ts_tablet_manager.cc:1403] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.026s user 0.000s sys 0.006s
I20260505 07:54:19.313863 1181 raft_consensus.cc:359] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.314714 1181 raft_consensus.cc:385] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.314921 1181 raft_consensus.cc:740] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.315275 1181 consensus_queue.cc:260] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.316654 1181 ts_tablet_manager.cc:1434] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260505 07:54:19.309257 1172 tablet_bootstrap.cc:654] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.321880 1181 tablet_bootstrap.cc:492] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.323499 1181 tablet_bootstrap.cc:654] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.329766 2276 ts_tablet_manager.cc:1929] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.329944 2276 log.cc:1199] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e7887c3462f74740b314e3bec36e280d
I20260505 07:54:19.330374 2276 ts_tablet_manager.cc:1950] T e7887c3462f74740b314e3bec36e280d P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:19.333323 2447 log.cc:1199] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/9044763bed1a41739ecce4def4cca1fe
I20260505 07:54:19.334049 2447 ts_tablet_manager.cc:1950] T 9044763bed1a41739ecce4def4cca1fe P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.335356 2268 ts_tablet_manager.cc:1929] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.340931 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 198) took 8583 ms. Trace:
I20260505 07:54:19.341146 2268 log.cc:1199] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/f200abef13ac42698a9b96fc8af82ff0
I20260505 07:54:19.341606 2268 ts_tablet_manager.cc:1950] T f200abef13ac42698a9b96fc8af82ff0 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:19.336279 1172 tablet_bootstrap.cc:492] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.341852 1172 ts_tablet_manager.cc:1403] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.000s
I20260505 07:54:19.314486 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49d7b9f5b6f74f8ea070f8ea7e545df7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.342373 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 201) took 9697 ms. Trace:
I20260505 07:54:19.342549 1172 raft_consensus.cc:359] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.342778 1172 raft_consensus.cc:385] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.342914 1172 raft_consensus.cc:740] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.343175 1172 consensus_queue.cc:260] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.343743 1172 ts_tablet_manager.cc:1434] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:19.344063 1172 tablet_bootstrap.cc:492] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.345242 2447 rpcz_store.cc:276] 0505 07:54:09.644951 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.645006 (+ 55us) service_pool.cc:224] Handling call
0505 07:54:19.232433 (+9587427us) ts_tablet_manager.cc:1151] Deleting tablet 9044763bed1a41739ecce4def4cca1fe
0505 07:54:19.232445 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.265505 (+ 33060us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.312750 (+ 47245us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.342353 (+ 29603us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9563708}
I20260505 07:54:19.345844 2447 tablet_replica.cc:333] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.347473 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 9044763bed1a41739ecce4def4cca1fe (table default.Table_1523b3ae7cd94c599e60a187bc677fc0 [id=bd575286f4e94df3b335b5e79d8f971c]) successfully deleted
I20260505 07:54:19.342656 782 raft_consensus.cc:3060] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.347888 2447 raft_consensus.cc:2243] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.348915 1172 tablet_bootstrap.cc:654] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.349499 2276 rpcz_store.cc:276] 0505 07:54:10.757747 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.757977 (+ 230us) service_pool.cc:224] Handling call
0505 07:54:19.225252 (+8467275us) ts_tablet_manager.cc:1151] Deleting tablet e7887c3462f74740b314e3bec36e280d
0505 07:54:19.225263 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.283072 (+ 57809us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.329710 (+ 46638us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.340911 (+ 11201us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8467075}
I20260505 07:54:19.349967 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e7887c3462f74740b314e3bec36e280d (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]) successfully deleted
I20260505 07:54:19.350286 2276 tablet_replica.cc:333] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:19.350463 2276 raft_consensus.cc:2243] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.350775 2276 raft_consensus.cc:2272] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.351341 2276 ts_tablet_manager.cc:1916] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.348066 2447 raft_consensus.cc:2272] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.350466 782 raft_consensus.cc:2468] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:19.352818 2447 ts_tablet_manager.cc:1916] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.352900 1181 tablet_bootstrap.cc:492] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.353031 1181 ts_tablet_manager.cc:1403] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.032s user 0.006s sys 0.000s
I20260505 07:54:19.353662 1181 raft_consensus.cc:359] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.353803 1181 raft_consensus.cc:385] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.353852 1181 raft_consensus.cc:740] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.354031 1181 consensus_queue.cc:260] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.354468 1181 ts_tablet_manager.cc:1434] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260505 07:54:19.352319 1176 tablet_bootstrap.cc:492] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.355173 1176 ts_tablet_manager.cc:1403] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.045s user 0.009s sys 0.003s
I20260505 07:54:19.355451 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 309) took 4687 ms. Trace:
I20260505 07:54:19.355533 2268 rpcz_store.cc:276] 0505 07:54:14.667873 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.668110 (+ 237us) service_pool.cc:224] Handling call
0505 07:54:19.239544 (+4571434us) ts_tablet_manager.cc:1151] Deleting tablet f200abef13ac42698a9b96fc8af82ff0
0505 07:54:19.239554 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.306070 (+ 66516us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.335308 (+ 29238us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.355431 (+ 20123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48772,"tablet-delete.queue_time_us":4568104}
I20260505 07:54:19.355773 1176 raft_consensus.cc:359] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.355911 1176 raft_consensus.cc:385] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.355957 1176 raft_consensus.cc:740] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.356014 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet f200abef13ac42698a9b96fc8af82ff0 (table default.Table_157815480b75424885451ec833e3438d [id=ebd83e4e101d4282ab8aeec21c281c60]) successfully deleted
I20260505 07:54:19.356145 1176 consensus_queue.cc:260] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.356590 1176 ts_tablet_manager.cc:1434] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260505 07:54:19.356805 1176 tablet_bootstrap.cc:492] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.358299 1176 tablet_bootstrap.cc:654] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.359571 1181 tablet_bootstrap.cc:492] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.361922 1181 tablet_bootstrap.cc:654] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.365442 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 310) took 4692 ms. Trace:
I20260505 07:54:19.365536 2268 rpcz_store.cc:276] 0505 07:54:14.673372 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.673418 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:19.365330 (+4691912us) ts_tablet_manager.cc:1151] Deleting tablet 472441bd29704be59aabb0f7a4964bb0
0505 07:54:19.365341 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.365424 (+ 83us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4691751}
I20260505 07:54:19.365952 2268 tablet_replica.cc:333] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
W20260505 07:54:19.365980 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 472441bd29704be59aabb0f7a4964bb0 because the tablet was not found. No further retry: Not found: Tablet not found: 472441bd29704be59aabb0f7a4964bb0
I20260505 07:54:19.366143 2268 raft_consensus.cc:2243] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.366444 2268 raft_consensus.cc:2272] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.367873 2268 ts_tablet_manager.cc:1916] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.373596 1172 tablet_bootstrap.cc:492] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.373785 1172 ts_tablet_manager.cc:1403] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20260505 07:54:19.374553 1172 raft_consensus.cc:359] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.374786 1172 raft_consensus.cc:385] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.374881 1172 raft_consensus.cc:740] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.375170 1172 consensus_queue.cc:260] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.376559 1172 ts_tablet_manager.cc:1434] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:19.387755 1172 tablet_bootstrap.cc:492] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.391122 1176 tablet_bootstrap.cc:492] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.391312 1176 ts_tablet_manager.cc:1403] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.010s
I20260505 07:54:19.392156 1176 raft_consensus.cc:359] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.392549 1176 raft_consensus.cc:385] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.392642 1176 raft_consensus.cc:740] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.392896 1176 consensus_queue.cc:260] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.393502 1176 ts_tablet_manager.cc:1434] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:19.394364 1181 tablet_bootstrap.cc:492] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.394537 1181 ts_tablet_manager.cc:1403] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.000s
I20260505 07:54:19.395274 1181 raft_consensus.cc:359] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.395473 1181 raft_consensus.cc:385] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.395562 1181 raft_consensus.cc:740] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.395824 1181 consensus_queue.cc:260] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.396399 1172 tablet_bootstrap.cc:654] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.393042 2447 ts_tablet_manager.cc:1929] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.400326 2447 log.cc:1199] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/cac5064f68d24ebe95b5a0b8e10f463e
I20260505 07:54:19.400807 2447 ts_tablet_manager.cc:1950] T cac5064f68d24ebe95b5a0b8e10f463e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.401521 2268 ts_tablet_manager.cc:1929] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.401628 2268 log.cc:1199] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/e7887c3462f74740b314e3bec36e280d
I20260505 07:54:19.402050 2268 ts_tablet_manager.cc:1950] T e7887c3462f74740b314e3bec36e280d P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:19.402830 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 311) took 4728 ms. Trace:
I20260505 07:54:19.402925 2268 rpcz_store.cc:276] 0505 07:54:14.674220 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.674258 (+ 38us) service_pool.cc:224] Handling call
0505 07:54:19.365909 (+4691651us) ts_tablet_manager.cc:1151] Deleting tablet e7887c3462f74740b314e3bec36e280d
0505 07:54:19.365918 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.398628 (+ 32710us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.401475 (+ 2847us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.402811 (+ 1336us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4691513}
I20260505 07:54:19.403239 1181 ts_tablet_manager.cc:1434] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.009s user 0.001s sys 0.000s
I20260505 07:54:19.403373 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet e7887c3462f74740b314e3bec36e280d (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]) successfully deleted
I20260505 07:54:19.403743 1181 tablet_bootstrap.cc:492] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.404762 1176 tablet_bootstrap.cc:492] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.408564 3261 consensus_queue.cc:1048] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.409480 1181 tablet_bootstrap.cc:654] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.409812 1176 tablet_bootstrap.cc:654] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.410046 2276 ts_tablet_manager.cc:1929] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.410172 2276 log.cc:1199] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/02e06d8815034d91986cef12964860ea
I20260505 07:54:19.410543 2276 ts_tablet_manager.cc:1950] T 02e06d8815034d91986cef12964860ea P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:19.412385 3177 raft_consensus.cc:493] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.412499 3177 raft_consensus.cc:515] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.412748 3296 raft_consensus.cc:493] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.412915 3296 raft_consensus.cc:515] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.413084 3177 leader_election.cc:290] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.413461 3296 leader_election.cc:290] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:19.414178 717 leader_election.cc:343] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: e010f23c4df647ab863e03c3cfed7898
I20260505 07:54:19.414366 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0fd88889b54aac88c4f18c0abe7a43" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.415177 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc0fd88889b54aac88c4f18c0abe7a43" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:19.416124 856 leader_election.cc:343] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.417181 3261 raft_consensus.cc:493] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260505 07:54:19.416450 854 leader_election.cc:343] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.418905 854 leader_election.cc:304] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.418147 3261 raft_consensus.cc:515] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.419337 3296 raft_consensus.cc:2749] T fc0fd88889b54aac88c4f18c0abe7a43 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.419770 3261 leader_election.cc:290] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.420416 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f840d5f58161468385d5c4f116b78c3d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.420562 907 raft_consensus.cc:2468] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.420961 991 leader_election.cc:304] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:19.421749 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f840d5f58161468385d5c4f116b78c3d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.421955 782 raft_consensus.cc:2468] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.422633 3261 raft_consensus.cc:2804] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.422704 3261 raft_consensus.cc:493] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.422755 3261 raft_consensus.cc:3060] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:19.423722 718 leader_election.cc:343] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: e010f23c4df647ab863e03c3cfed7898
I20260505 07:54:19.423831 718 leader_election.cc:304] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.425632 3261 raft_consensus.cc:515] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.426384 3199 raft_consensus.cc:2749] T e010f23c4df647ab863e03c3cfed7898 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260505 07:54:19.426860 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:19.427354 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f840d5f58161468385d5c4f116b78c3d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.427490 782 raft_consensus.cc:3060] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.427543 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:19.427982 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e243711164f04faabbde53b0a09d90b1 (table default.Table_a3cc6e739a71414f9f925aa759a07bd3 [id=5993b31e5350440f9715f1e416b90eb2]). Expected version 1 got 0
I20260505 07:54:19.428186 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:19.428483 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 03260a8507804e548486c81225de3ad9 (table default.Table_115e3886a51546409c5e744efd8b0462 [id=3c21c289d8f94c70b4ba7ef64dabc072]). Expected version 1 got 0
I20260505 07:54:19.428795 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:19.428965 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:19.429307 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:19.429806 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:19.429996 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:19.430159 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:19.430684 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:19.430718 782 raft_consensus.cc:2468] T f840d5f58161468385d5c4f116b78c3d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:19.430833 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:19.431190 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:19.431733 988 leader_election.cc:304] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.432056 3202 raft_consensus.cc:2804] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.432173 3202 raft_consensus.cc:697] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:19.434024 1172 tablet_bootstrap.cc:492] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.434155 1172 ts_tablet_manager.cc:1403] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.001s
I20260505 07:54:19.434399 3202 consensus_queue.cc:237] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.439749 1172 raft_consensus.cc:359] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.440004 1172 raft_consensus.cc:385] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.440111 1172 raft_consensus.cc:740] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.440387 1172 consensus_queue.cc:260] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.442018 1172 ts_tablet_manager.cc:1434] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260505 07:54:19.442471 3254 consensus_queue.cc:1048] T 68e8f9f7b30f4912af916c15f9ae25d8 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.442845 1172 tablet_bootstrap.cc:492] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:19.443490 991 consensus_peers.cc:597] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e8bf49b787ef4de095448751464708a4. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:19.445144 1181 tablet_bootstrap.cc:492] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.445314 1181 ts_tablet_manager.cc:1403] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.042s user 0.003s sys 0.004s
I20260505 07:54:19.446250 1181 raft_consensus.cc:359] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.446480 1181 raft_consensus.cc:385] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.446575 1181 raft_consensus.cc:740] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.446880 1181 consensus_queue.cc:260] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.447492 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 199) took 8689 ms. Trace:
I20260505 07:54:19.447546 1181 ts_tablet_manager.cc:1434] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:19.447896 1181 tablet_bootstrap.cc:492] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.449945 1181 tablet_bootstrap.cc:654] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.447604 2276 rpcz_store.cc:276] 0505 07:54:10.758164 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.758208 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:19.350237 (+8592029us) ts_tablet_manager.cc:1151] Deleting tablet 02e06d8815034d91986cef12964860ea
0505 07:54:19.350248 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.369678 (+ 19430us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.409987 (+ 40309us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.447461 (+ 37474us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8591891}
I20260505 07:54:19.451079 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 02e06d8815034d91986cef12964860ea (table default.Table_ca13a11f3fd249eabadc7213e7318acd [id=76602762055f42c0ad9805377eeb5d69]) successfully deleted
I20260505 07:54:19.451455 2276 tablet_replica.cc:333] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:19.452570 2276 raft_consensus.cc:2243] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.456765 2276 raft_consensus.cc:2272] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.457103 1172 tablet_bootstrap.cc:654] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.459197 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f840d5f58161468385d5c4f116b78c3d" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.459359 918 raft_consensus.cc:3060] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.459496 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 312) took 4783 ms. Trace:
I20260505 07:54:19.459571 2268 rpcz_store.cc:276] 0505 07:54:14.675534 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.675581 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:19.459397 (+4783816us) ts_tablet_manager.cc:1151] Deleting tablet 5ad8a79245ad47738d079752e51aed2b
0505 07:54:19.459408 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.459479 (+ 71us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4783639}
I20260505 07:54:19.459245 2276 ts_tablet_manager.cc:1916] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.461835 918 raft_consensus.cc:2468] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:19.462425 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 313) took 4784 ms. Trace:
W20260505 07:54:19.462594 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 5ad8a79245ad47738d079752e51aed2b because the tablet was not found. No further retry: Not found: Tablet not found: 5ad8a79245ad47738d079752e51aed2b
I20260505 07:54:19.462618 2268 rpcz_store.cc:276] 0505 07:54:14.677767 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.677809 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:19.462297 (+4784488us) ts_tablet_manager.cc:1151] Deleting tablet f2f5b72f58a24db0b58c15b47227bc60
0505 07:54:19.462309 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.462407 (+ 98us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4784329}
I20260505 07:54:19.463277 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 314) took 4785 ms. Trace:
W20260505 07:54:19.463343 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f2f5b72f58a24db0b58c15b47227bc60 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f2f5b72f58a24db0b58c15b47227bc60 already in progress: creating tablet
I20260505 07:54:19.463406 2268 rpcz_store.cc:276] 0505 07:54:14.678097 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.678136 (+ 39us) service_pool.cc:224] Handling call
0505 07:54:19.463195 (+4785059us) ts_tablet_manager.cc:1151] Deleting tablet e010f23c4df647ab863e03c3cfed7898
0505 07:54:19.463207 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.463263 (+ 56us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4784922}
I20260505 07:54:19.463855 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 315) took 4785 ms. Trace:
I20260505 07:54:19.463999 2268 rpcz_store.cc:276] 0505 07:54:14.678434 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.678477 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:19.463771 (+4785294us) ts_tablet_manager.cc:1151] Deleting tablet 1602ccbae24544c4a6cf14ea073880b4
0505 07:54:19.463780 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.463838 (+ 58us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4785140}
I20260505 07:54:19.464428 2268 tablet_replica.cc:333] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
W20260505 07:54:19.464708 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e010f23c4df647ab863e03c3cfed7898 because the tablet was not found. No further retry: Not found: Tablet not found: e010f23c4df647ab863e03c3cfed7898
W20260505 07:54:19.464962 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 1602ccbae24544c4a6cf14ea073880b4 because the tablet was not found. No further retry: Not found: Tablet not found: 1602ccbae24544c4a6cf14ea073880b4
I20260505 07:54:19.465426 3261 leader_election.cc:290] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.464990 2268 raft_consensus.cc:2243] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.465714 2268 raft_consensus.cc:2272] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.466178 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 202) took 9821 ms. Trace:
I20260505 07:54:19.466279 2447 rpcz_store.cc:276] 0505 07:54:09.645068 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.645145 (+ 77us) service_pool.cc:224] Handling call
0505 07:54:19.345793 (+9700648us) ts_tablet_manager.cc:1151] Deleting tablet cac5064f68d24ebe95b5a0b8e10f463e
0505 07:54:19.345805 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.355736 (+ 9931us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.392980 (+ 37244us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.466156 (+ 73176us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9677028}
I20260505 07:54:19.466662 2447 tablet_replica.cc:333] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.466917 2447 raft_consensus.cc:2243] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.467133 2447 raft_consensus.cc:2272] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.467537 1181 tablet_bootstrap.cc:492] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.467705 1181 ts_tablet_manager.cc:1403] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.002s
I20260505 07:54:19.468418 1181 raft_consensus.cc:359] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.468624 1181 raft_consensus.cc:385] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.468917 1181 raft_consensus.cc:740] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.469252 2276 ts_tablet_manager.cc:1929] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.469177 1181 consensus_queue.cc:260] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.469360 2276 log.cc:1199] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/a9b7c6e9463342a5a73af12da4447f89
I20260505 07:54:19.469700 2276 ts_tablet_manager.cc:1950] T a9b7c6e9463342a5a73af12da4447f89 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:19.469980 1181 ts_tablet_manager.cc:1434] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:19.470290 1181 tablet_bootstrap.cc:492] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.470857 2268 ts_tablet_manager.cc:1916] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.470094 2447 ts_tablet_manager.cc:1916] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.472442 1181 tablet_bootstrap.cc:654] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.477842 1172 tablet_bootstrap.cc:492] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.478031 1172 ts_tablet_manager.cc:1403] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.000s
I20260505 07:54:19.478698 1172 raft_consensus.cc:359] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.478929 1172 raft_consensus.cc:385] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.479029 1172 raft_consensus.cc:740] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.479331 1172 consensus_queue.cc:260] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.480702 1172 ts_tablet_manager.cc:1434] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:19.481566 1172 tablet_bootstrap.cc:492] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.483270 1172 tablet_bootstrap.cc:654] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.487690 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:19.488343 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:19.488775 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:19.489467 1176 tablet_bootstrap.cc:492] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.489513 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:19.489634 1176 ts_tablet_manager.cc:1403] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.085s user 0.003s sys 0.001s
I20260505 07:54:19.490568 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:19.491732 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 490483562d8f4c2daa7eaa333a9f3a0c (table default.Table_3acf7c3eb13d473a9d952098c7524753 [id=fe7ee6e8118840e8a9f70a007d80c2b3]). Expected version 1 got 0
I20260505 07:54:19.492506 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 200) took 8733 ms. Trace:
I20260505 07:54:19.492851 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:19.493314 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:19.493484 2276 rpcz_store.cc:276] 0505 07:54:10.758576 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.758621 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:19.451408 (+8692787us) ts_tablet_manager.cc:1151] Deleting tablet a9b7c6e9463342a5a73af12da4447f89
0505 07:54:19.451418 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.465487 (+ 14069us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.469202 (+ 3715us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.492481 (+ 23279us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8692650}
I20260505 07:54:19.494279 2276 tablet_replica.cc:333] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:19.494082 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet a9b7c6e9463342a5a73af12da4447f89 (table default.Table_0393e87d0a314ee898b11935e2f6666d [id=ab93bfbabbad4fad893f7a6dd595540f]) successfully deleted
I20260505 07:54:19.495209 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:19.495499 2276 raft_consensus.cc:2243] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.492611 3199 raft_consensus.cc:493] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.495724 2276 raft_consensus.cc:2272] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.495792 3199 raft_consensus.cc:515] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.496348 3199 leader_election.cc:290] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.490238 1176 raft_consensus.cc:359] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.496604 1176 raft_consensus.cc:385] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.496656 1176 raft_consensus.cc:740] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.496842 1176 consensus_queue.cc:260] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:19.497179 718 leader_election.cc:343] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 27bacc97d7534284970f218cf68ac75b
W20260505 07:54:19.497334 717 leader_election.cc:343] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 27bacc97d7534284970f218cf68ac75b
I20260505 07:54:19.497455 717 leader_election.cc:304] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.497557 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 89d3e08134704859abc44257f484b8cd (table default.Table_e9f1ff58ca6d41f19d05c458dc529941 [id=7579207045ec4220b966af9016a8f050]). Expected version 1 got 0
I20260505 07:54:19.497835 1176 ts_tablet_manager.cc:1434] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260505 07:54:19.498077 1176 tablet_bootstrap.cc:492] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.498046 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9926e569032144ec9d3148f4561333ab (table default.Table_3f9462e44d2b4d138eb0510c4a571080 [id=ed71bc1104b5423caedd9268977cc018]). Expected version 1 got 0
I20260505 07:54:19.498488 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 025c94c839484adba26945b62a217f5c (table default.Table_96e345e18b864cce8826bf32cac7ef52 [id=327b63fb4f744980a47bdce7db242629]). Expected version 1 got 0
I20260505 07:54:19.498688 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:19.498886 3199 raft_consensus.cc:2749] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.499974 1176 tablet_bootstrap.cc:654] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.500044 2268 ts_tablet_manager.cc:1929] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:19.500324 2276 ts_tablet_manager.cc:1916] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.493602 2447 ts_tablet_manager.cc:1929] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.503981 2447 log.cc:1199] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/a0f2f46d0e7840458cb992ce2a2dbea2
I20260505 07:54:19.504474 1172 tablet_bootstrap.cc:492] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.504590 2447 ts_tablet_manager.cc:1950] T a0f2f46d0e7840458cb992ce2a2dbea2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.504689 1172 ts_tablet_manager.cc:1403] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.023s user 0.002s sys 0.002s
I20260505 07:54:19.505409 1172 raft_consensus.cc:359] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.505641 1172 raft_consensus.cc:385] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.505754 1172 raft_consensus.cc:740] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.506028 1172 consensus_queue.cc:260] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.507421 1172 ts_tablet_manager.cc:1434] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:19.508279 1172 tablet_bootstrap.cc:492] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.509294 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 90612f0afb4949f491e625c61e40d247 (table default.Table_61a6fc82edd5408f991f40a8d15394a5 [id=9930805fdf364f25aea11f859b50bc71]). Expected version 1 got 0
I20260505 07:54:19.509336 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 203) took 9864 ms. Trace:
I20260505 07:54:19.509474 2268 log.cc:1199] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/6546745e894545ff9c5f91bcc6e1a41e
I20260505 07:54:19.509835 2268 ts_tablet_manager.cc:1950] T 6546745e894545ff9c5f91bcc6e1a41e P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:19.510154 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_1c26ba8978f749a38cfdb26ff048283c [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 2 got 0
I20260505 07:54:19.510455 2447 rpcz_store.cc:276] 0505 07:54:09.645167 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.645260 (+ 93us) service_pool.cc:224] Handling call
0505 07:54:19.466614 (+9821354us) ts_tablet_manager.cc:1151] Deleting tablet a0f2f46d0e7840458cb992ce2a2dbea2
0505 07:54:19.466623 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.479256 (+ 12633us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.493552 (+ 14296us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.509316 (+ 15764us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":668,"tablet-delete.queue_time_us":9797806}
I20260505 07:54:19.511119 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet a0f2f46d0e7840458cb992ce2a2dbea2 (table default.Table_f8f7a899e8c94e75958bd11fed643bc6 [id=a3cea9a70f4147b7bf5b630980c94c27]) successfully deleted
I20260505 07:54:19.511174 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 823cd847ae2b496ca7e99998297ba2fe (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:19.510008 1176 tablet_bootstrap.cc:492] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.511448 1176 ts_tablet_manager.cc:1403] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260505 07:54:19.511858 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:19.512068 1176 raft_consensus.cc:359] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.512231 1176 raft_consensus.cc:385] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.512298 1176 raft_consensus.cc:740] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.512466 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 316) took 4832 ms. Trace:
I20260505 07:54:19.512570 2447 tablet_replica.cc:333] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.512917 2447 raft_consensus.cc:2243] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.513298 2447 raft_consensus.cc:2272] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.513976 2447 ts_tablet_manager.cc:1916] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.514005 1172 tablet_bootstrap.cc:654] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.515699 2268 rpcz_store.cc:276] 0505 07:54:14.679520 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.679562 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:19.464380 (+4784818us) ts_tablet_manager.cc:1151] Deleting tablet 6546745e894545ff9c5f91bcc6e1a41e
0505 07:54:19.464391 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.486176 (+ 21785us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.499990 (+ 13814us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.512446 (+ 12456us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4784624}
I20260505 07:54:19.514667 1176 consensus_queue.cc:260] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.517549 1176 ts_tablet_manager.cc:1434] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260505 07:54:19.517814 1176 tablet_bootstrap.cc:492] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.519340 3296 raft_consensus.cc:493] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.519474 1176 tablet_bootstrap.cc:654] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.519495 3296 raft_consensus.cc:515] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.520020 3296 leader_election.cc:290] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:19.520745 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "994dbc9aa33248ee991dcf76018fe520" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:19.521232 854 leader_election.cc:343] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.521688 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "994dbc9aa33248ee991dcf76018fe520" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.522473 1181 tablet_bootstrap.cc:492] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.523088 1181 ts_tablet_manager.cc:1403] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.053s user 0.007s sys 0.005s
W20260505 07:54:19.523068 856 leader_election.cc:343] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.523164 856 leader_election.cc:304] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.523697 3296 raft_consensus.cc:2749] T 994dbc9aa33248ee991dcf76018fe520 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.524050 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 317) took 4843 ms. Trace:
I20260505 07:54:19.523054 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 6546745e894545ff9c5f91bcc6e1a41e (table default.Table_9c83e246b9a74343acdf478231329b39 [id=eca95f07408142a0a56860ba11d30a0f]) successfully deleted
I20260505 07:54:19.524721 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:19.524953 2268 rpcz_store.cc:276] 0505 07:54:14.680302 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.680348 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:19.523952 (+4843604us) ts_tablet_manager.cc:1151] Deleting tablet 727cacd242844455915bd9963d2db19f
0505 07:54:19.523962 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.524032 (+ 70us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4843423}
I20260505 07:54:19.524015 1181 raft_consensus.cc:359] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.525270 1181 raft_consensus.cc:385] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.525331 1181 raft_consensus.cc:740] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
W20260505 07:54:19.525362 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 727cacd242844455915bd9963d2db19f because the tablet was not found. No further retry: Not found: Tablet not found: 727cacd242844455915bd9963d2db19f
I20260505 07:54:19.525501 1181 consensus_queue.cc:260] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.526016 1181 ts_tablet_manager.cc:1434] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260505 07:54:19.526232 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1bd5951bfc3346c2a6aad8f33357394a (table default.Table_4e651b0d817f408e8e36b0399a57a04f [id=d85dae5f1f424012929e0d90ef8b2d22]). Expected version 1 got 0
I20260505 07:54:19.526806 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 27ab0ebc13f349f38cf86b9643700ae1 (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:19.526983 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a54d94d8a888483681858398f667a370 (table default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]). Expected version 3 got 0
I20260505 07:54:19.527251 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e38f248556bd4e49b06da72a1cbd7d8f (table default.Table_5c2b133923064d5eb9eabc21fb62f098 [id=12443115e63e4428a9cb798980fcbf5b]). Expected version 1 got 0
I20260505 07:54:19.527418 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_90c98f62112f465294c60c56f6835a93 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 2 got 0
I20260505 07:54:19.527832 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_65d14038399640c284b8382aa80e344f [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 2 got 0
I20260505 07:54:19.528102 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:19.528412 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1506cd31fe364e328867bbf70e36845e (table default.Table_eee3bd15017341fca226f671948e94e9 [id=c7cad0401e40485e9ed03160b9bed704]). Expected version 1 got 0
I20260505 07:54:19.528540 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:19.528733 2447 ts_tablet_manager.cc:1929] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.528853 2447 log.cc:1199] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/bf76e868bbcb44b0b1887c84dd27f093
I20260505 07:54:19.529131 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:19.529238 2447 ts_tablet_manager.cc:1950] T bf76e868bbcb44b0b1887c84dd27f093 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.529947 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 204) took 9884 ms. Trace:
I20260505 07:54:19.530030 2447 rpcz_store.cc:276] 0505 07:54:09.645270 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.645361 (+ 91us) service_pool.cc:224] Handling call
0505 07:54:19.512521 (+9867160us) ts_tablet_manager.cc:1151] Deleting tablet bf76e868bbcb44b0b1887c84dd27f093
0505 07:54:19.512532 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.525646 (+ 13114us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.528675 (+ 3029us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.529931 (+ 1256us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9842116}
I20260505 07:54:19.530455 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet bf76e868bbcb44b0b1887c84dd27f093 (table default.Table_d6881b2f97064289b8b333a795afa4ed [id=50b5f277bb144f4893d158d179bd0fb0]) successfully deleted
I20260505 07:54:19.530640 1181 tablet_bootstrap.cc:492] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.530879 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 319) took 4844 ms. Trace:
I20260505 07:54:19.530954 2268 rpcz_store.cc:276] 0505 07:54:14.686028 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.686066 (+ 38us) service_pool.cc:224] Handling call
0505 07:54:19.526178 (+4840112us) ts_tablet_manager.cc:1151] Deleting tablet 02412333633a4cdebe95a144fa6c363c
0505 07:54:19.530804 (+ 4626us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.530863 (+ 59us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4839987}
I20260505 07:54:19.531308 2268 tablet_replica.cc:333] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
W20260505 07:54:19.531299 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 02412333633a4cdebe95a144fa6c363c because the tablet was not found. No further retry: Not found: Tablet not found: 02412333633a4cdebe95a144fa6c363c
I20260505 07:54:19.532407 3254 raft_consensus.cc:493] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.532538 3254 raft_consensus.cc:515] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.533068 3254 leader_election.cc:290] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.533934 1172 tablet_bootstrap.cc:492] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.534094 1172 ts_tablet_manager.cc:1403] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.001s
I20260505 07:54:19.534133 1181 tablet_bootstrap.cc:654] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.534693 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172ca66c49354f549fa55fe63abf9970" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.534719 1172 raft_consensus.cc:359] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.534950 1172 raft_consensus.cc:385] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.535044 1172 raft_consensus.cc:740] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
W20260505 07:54:19.535081 988 leader_election.cc:343] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.535316 1172 consensus_queue.cc:260] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.536149 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172ca66c49354f549fa55fe63abf9970" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.539986 912 raft_consensus.cc:2468] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.540413 991 leader_election.cc:304] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.540578 3296 raft_consensus.cc:493] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.540686 3296 raft_consensus.cc:515] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.540946 3203 raft_consensus.cc:2804] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.541026 3203 raft_consensus.cc:493] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.541051 1176 tablet_bootstrap.cc:492] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.541082 3203 raft_consensus.cc:3060] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.541158 1176 ts_tablet_manager.cc:1403] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.002s
I20260505 07:54:19.541208 3296 leader_election.cc:290] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.541983 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02716f62aa354d8f839266661b7f05ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.542147 775 raft_consensus.cc:2468] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:19.542171 1176 raft_consensus.cc:359] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.542280 1176 raft_consensus.cc:385] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.542330 1176 raft_consensus.cc:740] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.542495 854 leader_election.cc:304] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.542522 1176 consensus_queue.cc:260] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.542718 3296 raft_consensus.cc:2804] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:19.542778 3296 raft_consensus.cc:493] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.542837 3296 raft_consensus.cc:3060] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:19.543087 1176 ts_tablet_manager.cc:1434] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:19.543308 1176 tablet_bootstrap.cc:492] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.543545 3203 raft_consensus.cc:515] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.544864 1176 tablet_bootstrap.cc:654] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.545336 3203 leader_election.cc:290] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.545428 3296 raft_consensus.cc:515] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.545712 2447 tablet_replica.cc:333] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.546002 2447 raft_consensus.cc:2243] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.545980 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172ca66c49354f549fa55fe63abf9970" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.546109 918 raft_consensus.cc:3060] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.546211 2447 raft_consensus.cc:2272] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.546686 2447 ts_tablet_manager.cc:1916] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:19.548463 856 leader_election.cc:343] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 02716f62aa354d8f839266661b7f05ea
I20260505 07:54:19.548540 918 raft_consensus.cc:2468] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:19.548772 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172ca66c49354f549fa55fe63abf9970" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.549183 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02716f62aa354d8f839266661b7f05ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.549238 991 leader_election.cc:304] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:19.549321 775 raft_consensus.cc:3060] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:19.550093 3254 raft_consensus.cc:2804] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.550201 3254 raft_consensus.cc:697] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:19.550411 3254 consensus_queue.cc:237] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:19.552469 988 leader_election.cc:343] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:19.553093 856 leader_election.cc:343] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 02716f62aa354d8f839266661b7f05ea
I20260505 07:54:19.553875 3203 consensus_queue.cc:1048] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.554162 775 raft_consensus.cc:2468] T 02716f62aa354d8f839266661b7f05ea P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 2.
I20260505 07:54:19.554316 3296 leader_election.cc:290] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.554765 854 leader_election.cc:304] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.555092 3298 raft_consensus.cc:2804] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:19.555183 3298 raft_consensus.cc:697] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:19.555389 3298 consensus_queue.cc:237] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.536327 1172 ts_tablet_manager.cc:1434] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:19.560510 1172 tablet_bootstrap.cc:492] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.561264 1176 tablet_bootstrap.cc:492] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.561390 1176 ts_tablet_manager.cc:1403] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.000s
I20260505 07:54:19.561602 2268 raft_consensus.cc:2243] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.562047 1176 raft_consensus.cc:359] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.562198 1176 raft_consensus.cc:385] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.562266 1176 raft_consensus.cc:740] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.562340 1172 tablet_bootstrap.cc:654] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.562484 1176 consensus_queue.cc:260] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.563850 1176 ts_tablet_manager.cc:1434] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:19.564667 1176 tablet_bootstrap.cc:492] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.566093 1176 tablet_bootstrap.cc:654] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.567646 2447 ts_tablet_manager.cc:1929] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.567847 2447 log.cc:1199] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/62d4bc3154704ba2bfde83bca10f812d
I20260505 07:54:19.568353 2447 ts_tablet_manager.cc:1950] T 62d4bc3154704ba2bfde83bca10f812d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.569043 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 205) took 9803 ms. Trace:
I20260505 07:54:19.569357 3202 raft_consensus.cc:493] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.569519 3202 raft_consensus.cc:515] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.570060 3202 leader_election.cc:290] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.570474 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d966bcd7f1419cbd090afc185ae78a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.570691 918 raft_consensus.cc:2468] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.571120 991 leader_election.cc:304] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:19.571424 3202 raft_consensus.cc:2804] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.571545 3202 raft_consensus.cc:493] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.571688 3202 raft_consensus.cc:3060] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.570355 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d966bcd7f1419cbd090afc185ae78a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.573181 2447 rpcz_store.cc:276] 0505 07:54:09.765827 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.765876 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:19.545669 (+9779793us) ts_tablet_manager.cc:1151] Deleting tablet 62d4bc3154704ba2bfde83bca10f812d
0505 07:54:19.545678 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.556907 (+ 11229us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.567550 (+ 10643us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.569027 (+ 1477us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9779592}
W20260505 07:54:19.573510 988 leader_election.cc:343] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.573645 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 62d4bc3154704ba2bfde83bca10f812d (table default.Table_82a545edb422426ab7eaeb4ae68a1a3f [id=5a1cf8871df74960bb29f3288ea4826c]) successfully deleted
I20260505 07:54:19.574260 2447 tablet_replica.cc:333] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.574396 3202 raft_consensus.cc:515] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.575181 1176 tablet_bootstrap.cc:492] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.575244 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d966bcd7f1419cbd090afc185ae78a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.575405 2447 raft_consensus.cc:2243] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.575294 1176 ts_tablet_manager.cc:1403] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.001s
W20260505 07:54:19.575712 988 leader_election.cc:343] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.575953 2447 raft_consensus.cc:2272] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.575974 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d966bcd7f1419cbd090afc185ae78a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.576100 918 raft_consensus.cc:3060] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.576210 1176 raft_consensus.cc:359] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.576336 1176 raft_consensus.cc:385] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.576380 1176 raft_consensus.cc:740] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.576409 2447 ts_tablet_manager.cc:1916] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.576565 1176 consensus_queue.cc:260] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.577117 1176 ts_tablet_manager.cc:1434] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260505 07:54:19.577489 1176 tablet_bootstrap.cc:492] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.578687 918 raft_consensus.cc:2468] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:19.579257 991 leader_election.cc:304] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.579569 3203 raft_consensus.cc:2804] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
W20260505 07:54:19.579684 717 consensus_peers.cc:597] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f842d8ac6b2142ffa69945c6087be34b. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:19.579850 3203 raft_consensus.cc:697] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:19.579792 1176 tablet_bootstrap.cc:654] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.580473 3203 consensus_queue.cc:237] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.584261 1172 tablet_bootstrap.cc:492] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.584378 1172 ts_tablet_manager.cc:1403] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.024s user 0.001s sys 0.005s
I20260505 07:54:19.584981 1172 raft_consensus.cc:359] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.585141 1172 raft_consensus.cc:385] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.585197 1172 raft_consensus.cc:740] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.585368 1172 consensus_queue.cc:260] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.586589 1172 ts_tablet_manager.cc:1434] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:19.586989 1172 tablet_bootstrap.cc:492] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.589128 1172 tablet_bootstrap.cc:654] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.591212 2276 ts_tablet_manager.cc:1929] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.591358 2276 log.cc:1199] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e82e9ae417784b7e83f4843b129d0580
I20260505 07:54:19.591804 2276 ts_tablet_manager.cc:1950] T e82e9ae417784b7e83f4843b129d0580 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:19.592552 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 201) took 8833 ms. Trace:
I20260505 07:54:19.592693 2276 rpcz_store.cc:276] 0505 07:54:10.759003 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.759280 (+ 277us) service_pool.cc:224] Handling call
0505 07:54:19.494234 (+8734954us) ts_tablet_manager.cc:1151] Deleting tablet e82e9ae417784b7e83f4843b129d0580
0505 07:54:19.494243 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.513355 (+ 19112us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.591160 (+ 77805us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.592534 (+ 1374us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8734813}
I20260505 07:54:19.593276 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e82e9ae417784b7e83f4843b129d0580 (table default.Table_9f8b49c83f7843318fd7a4645786e1d4 [id=a12c6d5dd696413490da09bf773c9a2f]) successfully deleted
I20260505 07:54:19.593771 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 202) took 8834 ms. Trace:
I20260505 07:54:19.593883 2276 rpcz_store.cc:276] 0505 07:54:10.759664 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.760526 (+ 862us) service_pool.cc:224] Handling call
0505 07:54:19.593681 (+8833155us) ts_tablet_manager.cc:1151] Deleting tablet accb2dedc30e494ebd22c33594cee661
0505 07:54:19.593690 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.593756 (+ 66us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8832983}
W20260505 07:54:19.594434 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet accb2dedc30e494ebd22c33594cee661 because the tablet was not found. No further retry: Not found: Tablet not found: accb2dedc30e494ebd22c33594cee661
I20260505 07:54:19.594760 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 203) took 8809 ms. Trace:
I20260505 07:54:19.594972 3202 leader_election.cc:290] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.598618 1176 tablet_bootstrap.cc:492] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.598750 1176 ts_tablet_manager.cc:1403] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.003s
I20260505 07:54:19.598968 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:19.599372 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:19.599360 1176 raft_consensus.cc:359] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.599524 1176 raft_consensus.cc:385] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.599601 1176 raft_consensus.cc:740] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.599879 2447 ts_tablet_manager.cc:1929] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.599841 1176 consensus_queue.cc:260] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.600312 2447 log.cc:1199] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/9959be9f83c2444282d012b61eef49b5
I20260505 07:54:19.600330 1358 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:19.600845 1176 ts_tablet_manager.cc:1434] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:19.601135 1176 tablet_bootstrap.cc:492] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.603045 2447 ts_tablet_manager.cc:1950] T 9959be9f83c2444282d012b61eef49b5 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.603160 1176 tablet_bootstrap.cc:654] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.603937 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 206) took 9838 ms. Trace:
I20260505 07:54:19.604995 2447 rpcz_store.cc:276] 0505 07:54:09.765965 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.766102 (+ 137us) service_pool.cc:224] Handling call
0505 07:54:19.574222 (+9808120us) ts_tablet_manager.cc:1151] Deleting tablet 9959be9f83c2444282d012b61eef49b5
0505 07:54:19.574229 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.591451 (+ 17222us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.599832 (+ 8381us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.603917 (+ 4085us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9808003}
I20260505 07:54:19.605484 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 9959be9f83c2444282d012b61eef49b5 (table default.Table_ae34ec4a8b52453dba7158ce065f4798 [id=8314d2bc7c1a401db1d6fac29fff5c3f]) successfully deleted
I20260505 07:54:19.605820 2447 tablet_replica.cc:333] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.606022 2447 raft_consensus.cc:2243] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.608417 2447 raft_consensus.cc:2272] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.609323 2447 ts_tablet_manager.cc:1916] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.620196 2447 ts_tablet_manager.cc:1929] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:19.621028 3173 consensus_queue.cc:1048] T 225c56ac920b448c89c38d6fb3c1bdef P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.623651 2447 log.cc:1199] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/8b57c1c618484d5d86a846fb982a2240
I20260505 07:54:19.624215 2447 ts_tablet_manager.cc:1950] T 8b57c1c618484d5d86a846fb982a2240 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.624927 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 207) took 9858 ms. Trace:
I20260505 07:54:19.627624 2447 rpcz_store.cc:276] 0505 07:54:09.766093 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.766245 (+ 152us) service_pool.cc:224] Handling call
0505 07:54:19.605752 (+9839507us) ts_tablet_manager.cc:1151] Deleting tablet 8b57c1c618484d5d86a846fb982a2240
0505 07:54:19.605774 (+ 22us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.616511 (+ 10737us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.620142 (+ 3631us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.624910 (+ 4768us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9839383}
I20260505 07:54:19.628140 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 8b57c1c618484d5d86a846fb982a2240 (table default.Table_041db4f8d9684d8e97ba6def427d554d [id=037074511cd8426487cfdb5298d1ee8e]) successfully deleted
I20260505 07:54:19.628589 2447 tablet_replica.cc:333] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.628899 2447 raft_consensus.cc:2243] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.629132 2447 raft_consensus.cc:2272] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.629823 2447 ts_tablet_manager.cc:1916] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.604664 1172 tablet_bootstrap.cc:492] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.636124 1172 ts_tablet_manager.cc:1403] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.000s
I20260505 07:54:19.636034 2276 rpcz_store.cc:276] 0505 07:54:10.797162 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.797209 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:19.594691 (+8797482us) ts_tablet_manager.cc:1151] Deleting tablet 54e4f7f895d943639653c8c32724baf2
0505 07:54:19.594699 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.594747 (+ 48us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8797306}
W20260505 07:54:19.636484 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 54e4f7f895d943639653c8c32724baf2 because the tablet was not found. No further retry: Not found: Tablet not found: 54e4f7f895d943639653c8c32724baf2
I20260505 07:54:19.636780 1172 raft_consensus.cc:359] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.636969 1172 raft_consensus.cc:385] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.637022 1172 raft_consensus.cc:740] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.637235 3177 raft_consensus.cc:493] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:19.637190 1172 consensus_queue.cc:260] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.637336 3177 raft_consensus.cc:515] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.637652 1172 ts_tablet_manager.cc:1434] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260505 07:54:19.637856 3177 leader_election.cc:290] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.637916 1172 tablet_bootstrap.cc:492] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:19.638703 717 leader_election.cc:343] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: a7d8265c05b74b28959d0de41fad1fde
I20260505 07:54:19.638860 3177 raft_consensus.cc:493] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:19.638976 3177 raft_consensus.cc:515] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.639477 3177 leader_election.cc:290] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.639554 1172 tablet_bootstrap.cc:654] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:19.640321 717 leader_election.cc:343] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 2d86b8b4dadf4fd2b6409a8cb70e313e
W20260505 07:54:19.641613 718 leader_election.cc:343] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 2d86b8b4dadf4fd2b6409a8cb70e313e
I20260505 07:54:19.641708 718 leader_election.cc:304] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.642089 3177 raft_consensus.cc:2749] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:19.642597 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 205) took 8819 ms. Trace:
I20260505 07:54:19.643158 2276 rpcz_store.cc:276] 0505 07:54:10.823389 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.823438 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:19.642508 (+8819070us) ts_tablet_manager.cc:1151] Deleting tablet 9044763bed1a41739ecce4def4cca1fe
0505 07:54:19.642519 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.642581 (+ 62us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8818889}
W20260505 07:54:19.643886 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 9044763bed1a41739ecce4def4cca1fe because the tablet was not found. No further retry: Not found: Tablet not found: 9044763bed1a41739ecce4def4cca1fe
I20260505 07:54:19.644330 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 204) took 8821 ms. Trace:
I20260505 07:54:19.647279 1181 tablet_bootstrap.cc:492] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.647482 1181 ts_tablet_manager.cc:1403] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.117s user 0.006s sys 0.004s
I20260505 07:54:19.647204 2276 rpcz_store.cc:276] 0505 07:54:10.822951 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.832070 (+ 9119us) service_pool.cc:224] Handling call
0505 07:54:19.643707 (+8811637us) ts_tablet_manager.cc:1151] Deleting tablet a7d8265c05b74b28959d0de41fad1fde
0505 07:54:19.644250 (+ 543us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.644313 (+ 63us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8811444}
W20260505 07:54:19.648044 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a7d8265c05b74b28959d0de41fad1fde because the tablet was not found. No further retry: Not found: Tablet not found: a7d8265c05b74b28959d0de41fad1fde
I20260505 07:54:19.648392 2276 tablet_replica.cc:333] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:19.648643 1181 raft_consensus.cc:359] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.649051 1181 raft_consensus.cc:385] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.649147 1181 raft_consensus.cc:740] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.649395 1181 consensus_queue.cc:260] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.649993 1181 ts_tablet_manager.cc:1434] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:19.650666 1181 tablet_bootstrap.cc:492] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.655535 2276 raft_consensus.cc:2243] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.655732 2276 raft_consensus.cc:2272] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.656337 2276 ts_tablet_manager.cc:1916] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.657433 1172 tablet_bootstrap.cc:492] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.657536 1172 ts_tablet_manager.cc:1403] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.006s
I20260505 07:54:19.658149 1172 raft_consensus.cc:359] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.658305 1172 raft_consensus.cc:385] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.658360 1172 raft_consensus.cc:740] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.658567 1172 consensus_queue.cc:260] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.658938 2447 ts_tablet_manager.cc:1929] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.659049 2447 log.cc:1199] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/feeafd614b3e4281a8e45b950e699b63
I20260505 07:54:19.659451 2447 ts_tablet_manager.cc:1950] T feeafd614b3e4281a8e45b950e699b63 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.660172 1172 ts_tablet_manager.cc:1434] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:19.660184 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 209) took 9893 ms. Trace:
I20260505 07:54:19.660254 2447 rpcz_store.cc:276] 0505 07:54:09.766345 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.766396 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:19.628528 (+9862132us) ts_tablet_manager.cc:1151] Deleting tablet feeafd614b3e4281a8e45b950e699b63
0505 07:54:19.628541 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.655783 (+ 27242us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.658888 (+ 3105us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.660166 (+ 1278us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9861993}
I20260505 07:54:19.660707 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet feeafd614b3e4281a8e45b950e699b63 (table default.Table_ad3e5b8653224a0c929e29e9daf47ff5 [id=99a81018fe1b45638089d579b76029d4]) successfully deleted
I20260505 07:54:19.661051 1181 tablet_bootstrap.cc:654] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.661451 1172 tablet_bootstrap.cc:492] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
W20260505 07:54:19.661393 718 leader_election.cc:343] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: a7d8265c05b74b28959d0de41fad1fde
I20260505 07:54:19.661530 718 leader_election.cc:304] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.662022 3177 raft_consensus.cc:2749] T a7d8265c05b74b28959d0de41fad1fde P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:19.662949 1172 tablet_bootstrap.cc:654] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.666090 1176 tablet_bootstrap.cc:492] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.666217 1176 ts_tablet_manager.cc:1403] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.065s user 0.010s sys 0.010s
I20260505 07:54:19.666839 1176 raft_consensus.cc:359] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.666997 1176 raft_consensus.cc:385] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.667068 1176 raft_consensus.cc:740] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.672109 2447 tablet_replica.cc:333] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.672405 3298 raft_consensus.cc:493] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.672431 3202 consensus_queue.cc:1048] T e8bf49b787ef4de095448751464708a4 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.672514 3298 raft_consensus.cc:515] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.673053 3298 leader_election.cc:290] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.674223 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7071df211bd34b86a751142bae3271b2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:19.674743 856 leader_election.cc:343] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.675386 3203 consensus_queue.cc:1048] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.675247 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7071df211bd34b86a751142bae3271b2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.676090 775 raft_consensus.cc:2468] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:19.676491 2447 raft_consensus.cc:2243] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.676542 854 leader_election.cc:304] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.676932 2447 raft_consensus.cc:2272] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.676895 3298 raft_consensus.cc:2804] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.677002 3298 raft_consensus.cc:493] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.677071 3298 raft_consensus.cc:3060] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.680594 3298 raft_consensus.cc:515] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.681520 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7071df211bd34b86a751142bae3271b2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.681629 3254 raft_consensus.cc:493] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.681696 776 raft_consensus.cc:3060] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.681875 3254 raft_consensus.cc:515] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.682602 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7071df211bd34b86a751142bae3271b2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
W20260505 07:54:19.683089 856 leader_election.cc:343] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.683970 3298 leader_election.cc:290] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.684211 776 raft_consensus.cc:2468] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:19.684607 854 leader_election.cc:304] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.684911 2447 ts_tablet_manager.cc:1916] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.685303 3296 raft_consensus.cc:2804] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.685406 3296 raft_consensus.cc:697] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:19.685606 3296 consensus_queue.cc:237] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.694664 1172 tablet_bootstrap.cc:492] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.694837 1172 ts_tablet_manager.cc:1403] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.000s
I20260505 07:54:19.695883 1172 raft_consensus.cc:359] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.672963 1176 consensus_queue.cc:260] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.696046 1172 raft_consensus.cc:385] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.696103 1172 raft_consensus.cc:740] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.696687 1176 ts_tablet_manager.cc:1434] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.030s user 0.001s sys 0.001s
I20260505 07:54:19.696313 1172 consensus_queue.cc:260] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.697292 1176 tablet_bootstrap.cc:492] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.698354 3199 raft_consensus.cc:493] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.698468 3199 raft_consensus.cc:515] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.698577 1172 ts_tablet_manager.cc:1434] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:19.699064 3199 leader_election.cc:290] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.699751 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1506cd31fe364e328867bbf70e36845e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.700335 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1506cd31fe364e328867bbf70e36845e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.700543 912 raft_consensus.cc:2393] T 1506cd31fe364e328867bbf70e36845e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:19.700616 3203 raft_consensus.cc:493] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.700743 3203 raft_consensus.cc:515] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.701105 718 leader_election.cc:304] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.701500 3199 raft_consensus.cc:3060] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.701839 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1d3829ede4b5582a022b64265df7e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:19.702301 988 leader_election.cc:343] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.701948 3203 leader_election.cc:290] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.703027 3004 raft_consensus.cc:493] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.703166 3004 raft_consensus.cc:515] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.703871 3004 leader_election.cc:290] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.703899 3199 raft_consensus.cc:2749] T 1506cd31fe364e328867bbf70e36845e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.704633 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310ef5aef7e44ec5a062a01d2a175a11" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.704787 1051 raft_consensus.cc:2468] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:19.704909 3004 raft_consensus.cc:493] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.705065 3004 raft_consensus.cc:515] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.705237 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310ef5aef7e44ec5a062a01d2a175a11" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.705375 918 raft_consensus.cc:2468] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:19.705713 3004 leader_election.cc:290] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.706216 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1d3829ede4b5582a022b64265df7e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.706362 912 raft_consensus.cc:2468] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.706669 991 leader_election.cc:304] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.706990 717 leader_election.cc:304] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.707059 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5eaa91ecc1f43d18b313b6be4cc211e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.707118 2447 ts_tablet_manager.cc:1929] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.707207 2447 log.cc:1199] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/0baf83b155494fddb3c25144608706f7
I20260505 07:54:19.707194 1049 raft_consensus.cc:2468] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:19.707289 3200 raft_consensus.cc:2804] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.707366 3200 raft_consensus.cc:493] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.707446 3200 raft_consensus.cc:3060] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.707679 2447 ts_tablet_manager.cc:1950] T 0baf83b155494fddb3c25144608706f7 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.708463 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 208) took 9942 ms. Trace:
I20260505 07:54:19.708539 2447 rpcz_store.cc:276] 0505 07:54:09.766220 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.766549 (+ 329us) service_pool.cc:224] Handling call
0505 07:54:19.672063 (+9905514us) ts_tablet_manager.cc:1151] Deleting tablet 0baf83b155494fddb3c25144608706f7
0505 07:54:19.672073 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.692193 (+ 20120us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.707064 (+ 14871us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.708445 (+ 1381us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9905378}
I20260505 07:54:19.708830 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:19.708923 2447 tablet_replica.cc:333] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.709038 3203 raft_consensus.cc:2804] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.709100 3203 raft_consensus.cc:493] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.709137 2447 raft_consensus.cc:2243] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.709156 3203 raft_consensus.cc:3060] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.709151 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:19.709376 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 03260a8507804e548486c81225de3ad9 (table default.Table_115e3886a51546409c5e744efd8b0462 [id=3c21c289d8f94c70b4ba7ef64dabc072]). Expected version 1 got 0
I20260505 07:54:19.709446 2447 raft_consensus.cc:2272] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.709658 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:19.709972 2447 ts_tablet_manager.cc:1916] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.710031 1172 tablet_bootstrap.cc:492] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.710312 3200 raft_consensus.cc:515] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.710937 3200 leader_election.cc:290] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.712232 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 0baf83b155494fddb3c25144608706f7 (table default.Table_5edc57ff557343629eb69ed0fe818ded [id=a92ad827e74542d98f9825d5aa198670]) successfully deleted
I20260505 07:54:19.712415 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310ef5aef7e44ec5a062a01d2a175a11" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:19.712551 1051 raft_consensus.cc:3060] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.712579 717 leader_election.cc:304] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.712612 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:19.712805 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:19.713519 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:19.713770 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "310ef5aef7e44ec5a062a01d2a175a11" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.713958 918 raft_consensus.cc:3060] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.714118 1172 tablet_bootstrap.cc:654] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.714826 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:19.714984 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:19.715132 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:19.715555 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet e243711164f04faabbde53b0a09d90b1 (table default.Table_a3cc6e739a71414f9f925aa759a07bd3 [id=5993b31e5350440f9715f1e416b90eb2]). Expected version 1 got 0
I20260505 07:54:19.715922 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:19.717281 918 raft_consensus.cc:2468] T 310ef5aef7e44ec5a062a01d2a175a11 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:19.717971 718 leader_election.cc:304] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.718322 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
W20260505 07:54:19.718345 718 leader_election.cc:343] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: e5eaa91ecc1f43d18b313b6be4cc211e
I20260505 07:54:19.718654 3298 consensus_queue.cc:1048] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.718849 3177 raft_consensus.cc:2804] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.718958 3177 raft_consensus.cc:493] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.719053 3177 raft_consensus.cc:3060] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.719687 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:19.720206 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:19.720953 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:19.721858 3177 raft_consensus.cc:515] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.722527 3177 leader_election.cc:290] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.721978 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:19.722690 3177 raft_consensus.cc:2804] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.723551 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
I20260505 07:54:19.724738 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 490483562d8f4c2daa7eaa333a9f3a0c (table default.Table_3acf7c3eb13d473a9d952098c7524753 [id=fe7ee6e8118840e8a9f70a007d80c2b3]). Expected version 1 got 0
I20260505 07:54:19.724939 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5eaa91ecc1f43d18b313b6be4cc211e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:19.725106 1047 raft_consensus.cc:3060] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.725276 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet e38f248556bd4e49b06da72a1cbd7d8f (table default.Table_5c2b133923064d5eb9eabc21fb62f098 [id=12443115e63e4428a9cb798980fcbf5b]). Expected version 1 got 0
W20260505 07:54:19.725607 718 leader_election.cc:343] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: e5eaa91ecc1f43d18b313b6be4cc211e
I20260505 07:54:19.725931 1350 catalog_manager.cc:5671] T 0bd2e2514c72419390c1bb9bd13f1fb3 P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:19.727118 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:19.728389 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_90c98f62112f465294c60c56f6835a93 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 2 got 0
I20260505 07:54:19.728940 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_1c26ba8978f749a38cfdb26ff048283c [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 2 got 0
I20260505 07:54:19.729333 2447 ts_tablet_manager.cc:1929] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.729457 2447 log.cc:1199] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/f92c53c68a4b4e319e98b84128e8e6a4
I20260505 07:54:19.729925 2447 ts_tablet_manager.cc:1950] T f92c53c68a4b4e319e98b84128e8e6a4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.730369 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:19.730746 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 211) took 9964 ms. Trace:
I20260505 07:54:19.730839 2447 rpcz_store.cc:276] 0505 07:54:09.766659 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.766701 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:19.708878 (+9942177us) ts_tablet_manager.cc:1151] Deleting tablet f92c53c68a4b4e319e98b84128e8e6a4
0505 07:54:19.708887 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.726075 (+ 17188us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.729276 (+ 3201us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.730729 (+ 1453us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9942024}
I20260505 07:54:19.731132 2447 tablet_replica.cc:333] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.731323 2447 raft_consensus.cc:2243] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.731299 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet f92c53c68a4b4e319e98b84128e8e6a4 (table default.Table_1496aa36a9c0478fa72f40d4595c2398 [id=73a1cc56dd244ba389866c21655a3a42]) successfully deleted
I20260505 07:54:19.730752 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:19.732153 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 90612f0afb4949f491e625c61e40d247 (table default.Table_61a6fc82edd5408f991f40a8d15394a5 [id=9930805fdf364f25aea11f859b50bc71]). Expected version 1 got 0
I20260505 07:54:19.732658 3254 leader_election.cc:290] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.733368 3200 raft_consensus.cc:493] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.733510 3200 raft_consensus.cc:515] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.734160 3200 leader_election.cc:290] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:19.734992 718 leader_election.cc:343] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 25072055f5534078adf5b5025614f951
I20260505 07:54:19.735172 3203 raft_consensus.cc:515] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.735271 3200 raft_consensus.cc:493] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:19.735455 3200 raft_consensus.cc:515] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.735730 1051 raft_consensus.cc:2468] T 310ef5aef7e44ec5a062a01d2a175a11 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:19.735772 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5eaa91ecc1f43d18b313b6be4cc211e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.735944 776 raft_consensus.cc:2393] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:19.736474 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25072055f5534078adf5b5025614f951" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.736125 3200 leader_election.cc:290] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.736601 1051 raft_consensus.cc:2468] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:19.736760 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1d3829ede4b5582a022b64265df7e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.736900 918 raft_consensus.cc:3060] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:19.737140 991 leader_election.cc:343] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: e5eaa91ecc1f43d18b313b6be4cc211e
I20260505 07:54:19.738209 1047 raft_consensus.cc:2468] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:19.739362 918 raft_consensus.cc:2468] T 6ea1d3829ede4b5582a022b64265df7e P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:19.739573 717 leader_election.cc:343] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: abe5f6f0774545b296bf1b03a08e145d
I20260505 07:54:19.739756 717 leader_election.cc:304] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:19.739956 3203 leader_election.cc:290] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.740134 717 leader_election.cc:304] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:19.740610 3199 raft_consensus.cc:2804] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.740715 3199 raft_consensus.cc:493] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.740833 3199 raft_consensus.cc:3060] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.741137 1181 tablet_bootstrap.cc:492] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.741286 1181 ts_tablet_manager.cc:1403] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.091s user 0.010s sys 0.014s
I20260505 07:54:19.741988 1181 raft_consensus.cc:359] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.742167 1181 raft_consensus.cc:385] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.742250 1181 raft_consensus.cc:740] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.743391 3199 raft_consensus.cc:515] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.743952 3199 leader_election.cc:290] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.744445 3199 raft_consensus.cc:2804] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.744428 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25072055f5534078adf5b5025614f951" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:19.744594 1047 raft_consensus.cc:3060] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.745486 3199 raft_consensus.cc:697] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:19.745796 3199 consensus_queue.cc:237] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.746724 988 leader_election.cc:304] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.747821 1047 raft_consensus.cc:2468] T 25072055f5534078adf5b5025614f951 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:19.749179 3254 raft_consensus.cc:2749] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.749348 991 leader_election.cc:304] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:19.749598 3254 raft_consensus.cc:2804] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.749703 3254 raft_consensus.cc:697] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:19.749922 3254 consensus_queue.cc:237] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.747563 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea1d3829ede4b5582a022b64265df7e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:19.751992 854 consensus_peers.cc:597] T 9ac029d9a3c540f58f393e79bf7be30e P d2799f5c6b4f4336b976ca9bbb458533 -> Peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Couldn't send request to peer eac7e1e8caba43cdb4df80cbb80e891c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:19.749042 718 leader_election.cc:343] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 25072055f5534078adf5b5025614f951
W20260505 07:54:19.752374 718 leader_election.cc:343] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: abe5f6f0774545b296bf1b03a08e145d
I20260505 07:54:19.752502 718 leader_election.cc:304] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.752883 3004 raft_consensus.cc:2749] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:19.753854 988 leader_election.cc:343] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.755091 717 leader_election.cc:304] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533
I20260505 07:54:19.755668 3200 raft_consensus.cc:2804] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.759858 1172 tablet_bootstrap.cc:492] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.759995 1172 ts_tablet_manager.cc:1403] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.050s user 0.004s sys 0.001s
I20260505 07:54:19.760628 1172 raft_consensus.cc:359] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.760779 1172 raft_consensus.cc:385] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.760828 1172 raft_consensus.cc:740] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.763342 1181 consensus_queue.cc:260] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.764271 1172 consensus_queue.cc:260] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.764766 1172 ts_tablet_manager.cc:1434] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:19.767552 3298 raft_consensus.cc:493] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.767681 3298 raft_consensus.cc:515] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.768204 3298 leader_election.cc:290] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:19.768306 3298 raft_consensus.cc:493] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.768390 3298 raft_consensus.cc:515] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.768762 3298 leader_election.cc:290] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:19.768816 1181 ts_tablet_manager.cc:1434] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.027s user 0.001s sys 0.001s
I20260505 07:54:19.769073 1181 tablet_bootstrap.cc:492] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.769507 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba1d19f902a4a0a8b581135f6611129" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.770067 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "212e496d564a4b5ebabb22504be76bf2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:19.770624 856 leader_election.cc:343] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.771199 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fba1d19f902a4a0a8b581135f6611129" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:19.771636 854 leader_election.cc:343] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.771840 1181 tablet_bootstrap.cc:654] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:19.771898 854 leader_election.cc:343] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.772004 854 leader_election.cc:304] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.772449 3298 raft_consensus.cc:2749] T fba1d19f902a4a0a8b581135f6611129 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.773326 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:19.774000 1172 tablet_bootstrap.cc:492] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.774735 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 823cd847ae2b496ca7e99998297ba2fe (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:19.775476 1172 tablet_bootstrap.cc:654] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.775731 2447 raft_consensus.cc:2272] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.776331 2447 ts_tablet_manager.cc:1916] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.776780 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 89d3e08134704859abc44257f484b8cd (table default.Table_e9f1ff58ca6d41f19d05c458dc529941 [id=7579207045ec4220b966af9016a8f050]). Expected version 1 got 0
I20260505 07:54:19.776953 3200 raft_consensus.cc:697] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:19.772737 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "212e496d564a4b5ebabb22504be76bf2" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:19.777700 856 leader_election.cc:343] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.777806 856 leader_election.cc:304] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.778211 3361 raft_consensus.cc:2749] T 212e496d564a4b5ebabb22504be76bf2 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.778301 3200 consensus_queue.cc:237] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.780282 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:19.782876 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 27ab0ebc13f349f38cf86b9643700ae1 (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:19.783185 2447 ts_tablet_manager.cc:1929] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.783280 2447 log.cc:1199] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/bc9ae6404dd64bd8bf4898768e94e4f1
I20260505 07:54:19.776861 3364 consensus_queue.cc:1048] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.784276 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 9926e569032144ec9d3148f4561333ab (table default.Table_3f9462e44d2b4d138eb0510c4a571080 [id=ed71bc1104b5423caedd9268977cc018]). Expected version 1 got 0
I20260505 07:54:19.785173 1181 tablet_bootstrap.cc:492] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.786031 1181 ts_tablet_manager.cc:1403] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20260505 07:54:19.786345 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_65d14038399640c284b8382aa80e344f [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 2 got 0
I20260505 07:54:19.787050 1181 raft_consensus.cc:359] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.787797 1172 tablet_bootstrap.cc:492] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.787907 1172 ts_tablet_manager.cc:1403] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.002s
I20260505 07:54:19.788493 1172 raft_consensus.cc:359] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.788631 1172 raft_consensus.cc:385] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.788674 1172 raft_consensus.cc:740] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.789526 1176 tablet_bootstrap.cc:654] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.787788 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:19.795358 1181 raft_consensus.cc:385] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.795650 1181 raft_consensus.cc:740] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.796286 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1506cd31fe364e328867bbf70e36845e (table default.Table_eee3bd15017341fca226f671948e94e9 [id=c7cad0401e40485e9ed03160b9bed704]). Expected version 1 got 0
I20260505 07:54:19.796536 1181 consensus_queue.cc:260] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.797011 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 0e08cf03ea0b495a94c6566ea6c8caad (table default.Table_696d4b08028248efafc4a12885184881 [id=4a85ae577239475da68f2677a043845e]). Expected version 1 got 0
I20260505 07:54:19.797755 1181 ts_tablet_manager.cc:1434] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20260505 07:54:19.798240 1350 catalog_manager.cc:5671] T a890d5e3f1af49b5aaa28c6c51cb3874 P eac7e1e8caba43cdb4df80cbb80e891c reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:19.799043 1181 tablet_bootstrap.cc:492] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.802611 2447 ts_tablet_manager.cc:1950] T bc9ae6404dd64bd8bf4898768e94e4f1 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.802845 3364 raft_consensus.cc:493] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.803243 3364 raft_consensus.cc:515] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.804090 3364 leader_election.cc:290] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.792033 1172 consensus_queue.cc:260] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.805197 2276 ts_tablet_manager.cc:1929] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.805301 2276 log.cc:1199] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/e6a5f823db6143319d743d363a77ea47
I20260505 07:54:19.805637 2276 ts_tablet_manager.cc:1950] T e6a5f823db6143319d743d363a77ea47 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:19.805971 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 212) took 10039 ms. Trace:
I20260505 07:54:19.806120 1172 ts_tablet_manager.cc:1434] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.018s user 0.001s sys 0.001s
I20260505 07:54:19.806344 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 206) took 8938 ms. Trace:
I20260505 07:54:19.806432 2276 rpcz_store.cc:276] 0505 07:54:10.868087 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.868136 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:19.647838 (+8779702us) ts_tablet_manager.cc:1151] Deleting tablet e6a5f823db6143319d743d363a77ea47
0505 07:54:19.648351 (+ 513us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.664503 (+ 16152us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.805143 (+140640us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.806325 (+ 1182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6916,"tablet-delete.queue_time_us":8779463}
I20260505 07:54:19.807353 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 207) took 8938 ms. Trace:
I20260505 07:54:19.807489 1172 tablet_bootstrap.cc:492] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.806497 2447 rpcz_store.cc:276] 0505 07:54:09.766809 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.766880 (+ 71us) service_pool.cc:224] Handling call
0505 07:54:19.731080 (+9964200us) ts_tablet_manager.cc:1151] Deleting tablet bc9ae6404dd64bd8bf4898768e94e4f1
0505 07:54:19.731090 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.780185 (+ 49095us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.783145 (+ 2960us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.805954 (+ 22809us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44225,"tablet-delete.queue_time_us":9964082}
I20260505 07:54:19.807854 2447 tablet_replica.cc:333] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.808068 2447 raft_consensus.cc:2243] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.808362 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09019fdf60374b11bf3427f21a55b5e0" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.808524 912 raft_consensus.cc:2468] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:19.808807 2447 raft_consensus.cc:2272] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.809089 1172 tablet_bootstrap.cc:654] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.807432 2276 rpcz_store.cc:276] 0505 07:54:10.869125 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.869179 (+ 54us) service_pool.cc:224] Handling call
0505 07:54:19.807267 (+8938088us) ts_tablet_manager.cc:1151] Deleting tablet 23fa401bb4634022bca3c6b23421e4d5
0505 07:54:19.807276 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.807336 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8937857}
I20260505 07:54:19.808852 718 leader_election.cc:304] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.810199 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 208) took 8938 ms. Trace:
I20260505 07:54:19.810279 2276 rpcz_store.cc:276] 0505 07:54:10.871551 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.871624 (+ 73us) service_pool.cc:224] Handling call
0505 07:54:19.810119 (+8938495us) ts_tablet_manager.cc:1151] Deleting tablet e278b86e7b5d444ebad8d0f6921d4c62
0505 07:54:19.810128 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.810184 (+ 56us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8938277}
I20260505 07:54:19.810768 2276 tablet_replica.cc:333] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:19.810952 2276 raft_consensus.cc:2243] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.811089 2276 raft_consensus.cc:2272] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.811439 2447 ts_tablet_manager.cc:1916] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.811527 2276 ts_tablet_manager.cc:1916] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.800768 3261 consensus_queue.cc:1048] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.807097 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet e6a5f823db6143319d743d363a77ea47 (table default.Table_61a6fc82edd5408f991f40a8d15394a5 [id=9930805fdf364f25aea11f859b50bc71]) successfully deleted
W20260505 07:54:19.816679 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 23fa401bb4634022bca3c6b23421e4d5 because the tablet was not found. No further retry: Not found: Tablet not found: 23fa401bb4634022bca3c6b23421e4d5
W20260505 07:54:19.817153 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e278b86e7b5d444ebad8d0f6921d4c62 because the tablet was not found. No further retry: Not found: Tablet not found: e278b86e7b5d444ebad8d0f6921d4c62
W20260505 07:54:19.814378 717 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.105.3:42069, user_credentials={real_user=slave}} blocked reactor thread for 58873.9us
I20260505 07:54:19.817907 3199 raft_consensus.cc:2804] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.817950 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet bc9ae6404dd64bd8bf4898768e94e4f1 (table default.Table_4060b1611196402a829ecc88ff74d21f [id=feb89d1a30de44fab54f4b8cc700735b]) successfully deleted
I20260505 07:54:19.818070 3199 raft_consensus.cc:493] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.818142 3199 raft_consensus.cc:3060] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.818753 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09019fdf60374b11bf3427f21a55b5e0" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.819078 3004 consensus_queue.cc:1048] T f36a0f68fc2b4adda04ec3b9db5f2d8e P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:19.819139 1051 raft_consensus.cc:2468] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:19.820566 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 6f207a3ffbc94b01abd9e9d6837f5c81 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]). Expected version 1 got 0
I20260505 07:54:19.822201 1181 tablet_bootstrap.cc:654] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.822852 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 025c94c839484adba26945b62a217f5c (table default.Table_96e345e18b864cce8826bf32cac7ef52 [id=327b63fb4f744980a47bdce7db242629]). Expected version 1 got 0
I20260505 07:54:19.824642 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3462dcc6391741e59dcb5cd08527e50f (table default.Table_0bbafa19896846f1abb669a5110cfe5e [id=509c7b9d25aa48af9dde3825cfe313d0]). Expected version 2 got 0
I20260505 07:54:19.825083 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 3ba4533c78974853b48526c465e38941 (table default.Table_9b4d72020d274a4792c92e9350519112 [id=618b6db3796e4213b89137cf012f89d0]). Expected version 1 got 0
I20260505 07:54:19.825120 3004 raft_consensus.cc:493] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.825651 3004 raft_consensus.cc:515] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.827126 3004 leader_election.cc:290] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.827246 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:19.827489 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:19.827754 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 1bd5951bfc3346c2a6aad8f33357394a (table default.Table_4e651b0d817f408e8e36b0399a57a04f [id=d85dae5f1f424012929e0d90ef8b2d22]). Expected version 1 got 0
I20260505 07:54:19.828012 1350 catalog_manager.cc:5763] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:19.828313 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efb07cb60f64dfead739a3ca1343df9" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.829196 3199 raft_consensus.cc:515] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.829272 1353 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 265) took 1292 ms. Trace:
I20260505 07:54:19.829365 1353 rpcz_store.cc:276] 0505 07:54:18.536426 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.714485 (+178059us) service_pool.cc:224] Handling call
0505 07:54:18.714845 (+ 360us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:18.715094 (+ 249us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:18.715943 (+ 849us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:18.716029 (+ 86us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:18.737696 (+ 21667us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:19.829201 (+1091505us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:18.716506 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:18.716593 (+ 87us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:18.716599 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:18.716601 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:18.716694 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:18.716707 (+ 13us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:18.716711 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:18.716770 (+ 59us) tablet.cc:701] Row locks acquired
0505 07:54:18.716772 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:18.716803 (+ 31us) write_op.cc:270] Start()
0505 07:54:18.716833 (+ 30us) write_op.cc:276] Timestamp: P: 1777967658716753 usec, L: 0
0505 07:54:18.716837 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:18.716996 (+ 159us) log.cc:844] Serialized 405 byte log entry
0505 07:54:18.734098 (+ 17102us) op_driver.cc:464] REPLICATION: finished
0505 07:54:18.734238 (+ 140us) write_op.cc:301] APPLY: starting
0505 07:54:18.734255 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:18.734294 (+ 39us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:18.734325 (+ 31us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:18.734376 (+ 51us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:18.734408 (+ 32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:18.734416 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:18.734488 (+ 72us) log.cc:844] Serialized 23 byte log entry
0505 07:54:18.734550 (+ 62us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:18.734570 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:18.734578 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:18.734652 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:18.734658 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":94,"apply.run_cpu_time_us":406,"apply.run_wall_time_us":406,"num_ops":1,"prepare.queue_time_us":46,"prepare.run_cpu_time_us":599,"prepare.run_wall_time_us":599,"raft.queue_time_us":704,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":188,"replication_time_us":17233}]]}
I20260505 07:54:19.830494 1353 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_79892c60ca2c461087524d2724724ab6" } modify_external_catalogs: true
I20260505 07:54:19.830647 1353 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_79892c60ca2c461087524d2724724ab6" } modify_external_catalogs: true
I20260505 07:54:19.831411 971 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33565
I20260505 07:54:19.831564 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:19.832165 1102 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33565
I20260505 07:54:19.832304 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:19.836304 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efb07cb60f64dfead739a3ca1343df9" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.836549 918 raft_consensus.cc:2468] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:19.838176 3203 raft_consensus.cc:493] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.838376 3203 raft_consensus.cc:515] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.839758 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01431ad112e8445487823efe8feb0dbe" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.839987 918 raft_consensus.cc:2468] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.840714 991 leader_election.cc:304] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:19.841090 3261 raft_consensus.cc:2804] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.841181 3261 raft_consensus.cc:493] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.841259 3261 raft_consensus.cc:3060] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.844170 3261 raft_consensus.cc:515] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.844825 3261 leader_election.cc:290] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.845463 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01431ad112e8445487823efe8feb0dbe" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.845750 918 raft_consensus.cc:3060] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.847654 3203 leader_election.cc:290] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.849591 918 raft_consensus.cc:2468] T 01431ad112e8445487823efe8feb0dbe P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:19.851046 991 leader_election.cc:304] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:19.851342 3203 raft_consensus.cc:2804] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.851471 3203 raft_consensus.cc:697] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:19.851734 3203 consensus_queue.cc:237] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:19.829479 717 leader_election.cc:343] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.857549 718 leader_election.cc:304] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.858634 3363 raft_consensus.cc:2804] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.858785 3363 raft_consensus.cc:493] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.858919 3363 raft_consensus.cc:3060] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.859073 829 heartbeater.cc:344] Connected to a master server at 127.0.105.61:33565
I20260505 07:54:19.859117 1176 tablet_bootstrap.cc:492] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.859238 829 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:19.859288 1176 ts_tablet_manager.cc:1403] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.162s user 0.001s sys 0.001s
I20260505 07:54:19.862938 3296 raft_consensus.cc:493] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.863073 3296 raft_consensus.cc:515] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.863852 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba4533c78974853b48526c465e38941" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:19.864295 856 leader_election.cc:343] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.864336 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ba4533c78974853b48526c465e38941" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.864542 3296 leader_election.cc:290] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:19.864770 854 leader_election.cc:343] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.864848 854 leader_election.cc:304] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.865172 3144 raft_consensus.cc:2749] T 3ba4533c78974853b48526c465e38941 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.864647 3296 raft_consensus.cc:493] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.865345 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09019fdf60374b11bf3427f21a55b5e0" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:19.865489 1047 raft_consensus.cc:3060] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.865540 3144 raft_consensus.cc:493] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.865622 3144 raft_consensus.cc:515] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.866065 3144 leader_election.cc:290] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.866600 3296 raft_consensus.cc:515] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:19.867803 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e608ac01ef4a88906e585bd7ee25ac" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.867972 775 raft_consensus.cc:2468] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:19.868201 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73042a7ff5fb43f7a781b4f98f4f4020" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.868323 854 leader_election.cc:304] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.868674 3144 raft_consensus.cc:2804] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.868755 3144 raft_consensus.cc:493] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.868827 3144 raft_consensus.cc:3060] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.871876 3296 leader_election.cc:290] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:19.872128 854 leader_election.cc:343] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.872829 3144 raft_consensus.cc:515] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.873653 3144 leader_election.cc:290] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.874033 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e608ac01ef4a88906e585bd7ee25ac" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:19.874076 1349 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 429ccdb56f0b456483cf1debb5de0beb
I20260505 07:54:19.874202 1048 raft_consensus.cc:3060] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.874369 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e608ac01ef4a88906e585bd7ee25ac" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.874516 776 raft_consensus.cc:3060] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.874667 1349 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:46452 (request call id 262) took 1384 ms. Trace:
I20260505 07:54:19.874763 1349 rpcz_store.cc:276] 0505 07:54:18.489955 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.491142 (+ 1187us) service_pool.cc:224] Handling call
0505 07:54:18.491703 (+ 561us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:18.491866 (+ 163us) catalog_manager.cc:2815] Modifying in-memory table state
0505 07:54:18.491926 (+ 60us) catalog_manager.cc:2824] Locking tablets
0505 07:54:18.492077 (+ 151us) catalog_manager.cc:2838] Removing table and tablets from system table
0505 07:54:18.511372 (+ 19295us) catalog_manager.cc:2856] Removing table from by-name map
0505 07:54:18.707492 (+196120us) catalog_manager.cc:2872] Committing in-memory state
0505 07:54:19.874017 (+1166525us) catalog_manager.cc:2876] Aborting table tasks
0505 07:54:19.874605 (+ 588us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:18.496997 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:18.497109 (+ 112us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:18.497116 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:18.497118 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:18.497231 (+ 113us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:18.497240 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:18.497244 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:18.497341 (+ 97us) tablet.cc:701] Row locks acquired
0505 07:54:18.497343 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:18.497371 (+ 28us) write_op.cc:270] Start()
0505 07:54:18.497398 (+ 27us) write_op.cc:276] Timestamp: P: 1777967658497344 usec, L: 0
0505 07:54:18.497401 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:18.497546 (+ 145us) log.cc:844] Serialized 765 byte log entry
0505 07:54:18.508236 (+ 10690us) op_driver.cc:464] REPLICATION: finished
0505 07:54:18.508860 (+ 624us) write_op.cc:301] APPLY: starting
0505 07:54:18.508869 (+ 9us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:18.508912 (+ 43us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:18.508933 (+ 21us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:18.508981 (+ 48us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:18.509017 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0505 07:54:18.509024 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:18.509073 (+ 49us) log.cc:844] Serialized 29 byte log entry
0505 07:54:18.509108 (+ 35us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:18.509120 (+ 12us) write_op.cc:454] Released schema lock
0505 07:54:18.509125 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:18.509178 (+ 53us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:18.509181 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":592,"apply.run_cpu_time_us":312,"apply.run_wall_time_us":311,"num_ops":2,"prepare.queue_time_us":4355,"prepare.run_cpu_time_us":640,"prepare.run_wall_time_us":988,"raft.queue_time_us":40,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":6,"replication_time_us":10810}]]}
I20260505 07:54:19.875659 1349 catalog_manager.cc:3544] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46452:
table { table_name: "default.Table_e137326bc9684662bffdc5c59e122c6a" } new_table_name: "default.Table_647fec9e0fb846cd86a882ea54f6a79a" modify_external_catalogs: true
I20260505 07:54:19.877040 776 raft_consensus.cc:2468] T 68e608ac01ef4a88906e585bd7ee25ac P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:19.877449 854 leader_election.cc:304] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.877966 3361 raft_consensus.cc:2804] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.878105 3361 raft_consensus.cc:697] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:19.878379 3361 consensus_queue.cc:237] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.880239 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 429ccdb56f0b456483cf1debb5de0beb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:19.881407 745 tablet_service.cc:1558] Processing DeleteTablet for tablet 429ccdb56f0b456483cf1debb5de0beb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:19.882213 717 consensus_peers.cc:597] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df1b62a30b2842b897a04c793bb5e7ce. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:19.883365 2276 ts_tablet_manager.cc:1929] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:19.884935 1176 raft_consensus.cc:359] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.885238 1176 raft_consensus.cc:385] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.885356 1176 raft_consensus.cc:740] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.885679 1176 consensus_queue.cc:260] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.886368 3199 leader_election.cc:290] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.886592 2276 log.cc:1199] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/b12d2d2b866348d781209fa787da7d7e
I20260505 07:54:19.886956 2276 ts_tablet_manager.cc:1950] T b12d2d2b866348d781209fa787da7d7e P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:19.887056 1176 ts_tablet_manager.cc:1434] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.028s user 0.002s sys 0.000s
W20260505 07:54:19.887344 1354 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 24f4854e8752493fb832ae2c12a033ce consensus state has no leader
I20260505 07:54:19.887513 1354 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 270) took 1303 ms. Trace:
I20260505 07:54:19.887701 1176 tablet_bootstrap.cc:492] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:19.887634 1354 rpcz_store.cc:276] 0505 07:54:18.583654 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.973169 (+389515us) service_pool.cc:224] Handling call
0505 07:54:18.973623 (+ 454us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:18.973898 (+ 275us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:19.037603 (+ 63705us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:19.037747 (+ 144us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:19.045714 (+ 7967us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:19.887483 (+841769us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:19.038489 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:19.038612 (+ 123us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:19.038619 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:19.038621 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:19.038731 (+ 110us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:19.038749 (+ 18us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:19.038757 (+ 8us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:19.038834 (+ 77us) tablet.cc:701] Row locks acquired
0505 07:54:19.038836 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:19.038876 (+ 40us) write_op.cc:270] Start()
0505 07:54:19.038927 (+ 51us) write_op.cc:276] Timestamp: P: 1777967659038838 usec, L: 0
0505 07:54:19.038931 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:19.039143 (+ 212us) log.cc:844] Serialized 405 byte log entry
0505 07:54:19.043536 (+ 4393us) op_driver.cc:464] REPLICATION: finished
0505 07:54:19.043786 (+ 250us) write_op.cc:301] APPLY: starting
0505 07:54:19.043805 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:19.043869 (+ 64us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:19.043903 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:19.043961 (+ 58us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:19.043999 (+ 38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:19.044007 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:19.044091 (+ 84us) log.cc:844] Serialized 23 byte log entry
0505 07:54:19.044158 (+ 67us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:19.044177 (+ 19us) write_op.cc:454] Released schema lock
0505 07:54:19.044184 (+ 7us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:19.044261 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:19.044267 (+ 6us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":189,"apply.run_cpu_time_us":467,"apply.run_wall_time_us":467,"num_ops":1,"prepare.queue_time_us":60,"prepare.run_cpu_time_us":789,"prepare.run_wall_time_us":789,"raft.queue_time_us":190,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":237,"replication_time_us":4569}]]}
I20260505 07:54:19.888260 1354 master_service.cc:954] ReplaceTablet: received request to replace tablet 75de7522fa534140822ead8be4dc2b2d from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:19.889109 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 209) took 8966 ms. Trace:
W20260505 07:54:19.889428 1357 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0ffcea5808c34be595bd870c0b0f6e10 consensus state has no leader
I20260505 07:54:19.889580 1357 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 269) took 1319 ms. Trace:
I20260505 07:54:19.889685 1357 rpcz_store.cc:276] 0505 07:54:18.569867 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.826958 (+257091us) service_pool.cc:224] Handling call
0505 07:54:18.827348 (+ 390us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:18.827533 (+ 185us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:18.827700 (+ 167us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:18.827787 (+ 87us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:19.002399 (+174612us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:19.889549 (+887150us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:18.828215 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:18.828329 (+ 114us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:18.828335 (+ 6us) write_op.cc:435] Acquired schema lock
0505 07:54:18.828337 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:18.828416 (+ 79us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:18.828425 (+ 9us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:18.828429 (+ 4us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:18.828478 (+ 49us) tablet.cc:701] Row locks acquired
0505 07:54:18.828480 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:18.828506 (+ 26us) write_op.cc:270] Start()
0505 07:54:18.828531 (+ 25us) write_op.cc:276] Timestamp: P: 1777967658828447 usec, L: 0
0505 07:54:18.828534 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:18.828661 (+ 127us) log.cc:844] Serialized 405 byte log entry
0505 07:54:19.000495 (+171834us) op_driver.cc:464] REPLICATION: finished
0505 07:54:19.001165 (+ 670us) write_op.cc:301] APPLY: starting
0505 07:54:19.001177 (+ 12us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:19.001205 (+ 28us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:19.001226 (+ 21us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:19.001260 (+ 34us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:19.001285 (+ 25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:19.001292 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:19.001346 (+ 54us) log.cc:844] Serialized 23 byte log entry
0505 07:54:19.001383 (+ 37us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:19.001396 (+ 13us) write_op.cc:454] Released schema lock
0505 07:54:19.001402 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:19.001457 (+ 55us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:19.001461 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":636,"apply.run_cpu_time_us":288,"apply.run_wall_time_us":287,"num_ops":1,"prepare.queue_time_us":43,"prepare.run_cpu_time_us":536,"prepare.run_wall_time_us":536,"raft.queue_time_us":305,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"replication_time_us":171942}]]}
I20260505 07:54:19.890408 3363 raft_consensus.cc:515] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.890550 1357 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_06e4cf6d3b7b4b8489fba5900699c2dd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:19.890864 718 consensus_peers.cc:597] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df1b62a30b2842b897a04c793bb5e7ce. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:19.891058 718 consensus_peers.cc:597] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f3e076892234c1494b1a5c13cbfd2e2. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:19.891137 1357 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_06e4cf6d3b7b4b8489fba5900699c2dd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:19.893042 1351 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_c3436374329a44868874945e540a1d9b [id=a91d46106bbd4967ad97e5de33b59891] alter complete (version 1)
I20260505 07:54:19.895817 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01431ad112e8445487823efe8feb0dbe" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.896586 778 raft_consensus.cc:3060] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.899128 2276 rpcz_store.cc:276] 0505 07:54:10.923156 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.924008 (+ 852us) service_pool.cc:224] Handling call
0505 07:54:19.810728 (+8886720us) ts_tablet_manager.cc:1151] Deleting tablet b12d2d2b866348d781209fa787da7d7e
0505 07:54:19.810736 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.853990 (+ 43254us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.883313 (+ 29323us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.889092 (+ 5779us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8886539}
I20260505 07:54:19.902482 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01431ad112e8445487823efe8feb0dbe" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.907296 1176 tablet_bootstrap.cc:654] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:19.907979 1355 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 59a3cd77808c4e858ea2ef9ed2a28c38 consensus state has no leader
I20260505 07:54:19.908155 1355 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 268) took 1341 ms. Trace:
I20260505 07:54:19.908239 1355 rpcz_store.cc:276] 0505 07:54:18.566347 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.824298 (+257951us) service_pool.cc:224] Handling call
0505 07:54:18.824869 (+ 571us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:18.825121 (+ 252us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:18.825277 (+ 156us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:18.825352 (+ 75us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:19.002353 (+177001us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:19.908104 (+905751us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:18.825941 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:18.826028 (+ 87us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:18.826036 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:18.826038 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:18.826150 (+ 112us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:18.826164 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:18.826169 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:18.826234 (+ 65us) tablet.cc:701] Row locks acquired
0505 07:54:18.826236 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:18.826270 (+ 34us) write_op.cc:270] Start()
0505 07:54:18.826306 (+ 36us) write_op.cc:276] Timestamp: P: 1777967658826208 usec, L: 0
0505 07:54:18.826310 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:18.826485 (+ 175us) log.cc:844] Serialized 405 byte log entry
0505 07:54:19.000413 (+173928us) op_driver.cc:464] REPLICATION: finished
0505 07:54:19.000592 (+ 179us) write_op.cc:301] APPLY: starting
0505 07:54:19.000613 (+ 21us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:19.000690 (+ 77us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:19.000724 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:19.000781 (+ 57us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:19.000817 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:19.000824 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:19.000919 (+ 95us) log.cc:844] Serialized 23 byte log entry
0505 07:54:19.000987 (+ 68us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:19.001008 (+ 21us) write_op.cc:454] Released schema lock
0505 07:54:19.001016 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:19.001090 (+ 74us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:19.001094 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":123,"apply.run_cpu_time_us":488,"apply.run_wall_time_us":487,"num_ops":1,"prepare.queue_time_us":102,"prepare.run_cpu_time_us":653,"prepare.run_wall_time_us":654,"raft.queue_time_us":2232,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":201,"replication_time_us":174067}]]}
I20260505 07:54:19.909013 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_7668619b64534841ad09278adaf5a613"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:19.909651 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7668619b64534841ad09278adaf5a613 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:19.899633 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09019fdf60374b11bf3427f21a55b5e0" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.910202 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 211) took 8968 ms. Trace:
I20260505 07:54:19.910264 918 raft_consensus.cc:3060] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.913324 918 raft_consensus.cc:2468] T 09019fdf60374b11bf3427f21a55b5e0 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:19.900923 3361 raft_consensus.cc:493] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.913707 3361 raft_consensus.cc:515] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.915303 3361 leader_election.cc:290] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.915375 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8e6019406845ebb29051432a96dfd1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.915585 1050 raft_consensus.cc:2468] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
I20260505 07:54:19.916152 856 leader_election.cc:304] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:19.916672 3361 raft_consensus.cc:2804] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.916755 3361 raft_consensus.cc:493] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.916841 3361 raft_consensus.cc:3060] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.919335 3361 raft_consensus.cc:515] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.920878 3361 leader_election.cc:290] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.920966 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8e6019406845ebb29051432a96dfd1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:19.921103 1050 raft_consensus.cc:3060] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.922369 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 429ccdb56f0b456483cf1debb5de0beb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-05-05 07:54:18 UTC) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:19.923347 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet b12d2d2b866348d781209fa787da7d7e (table default.Table_7ed6d926da574429be8db03bde4b4221 [id=e52c3ee18ce644d2b988c2e70c6307d7]) successfully deleted
I20260505 07:54:19.915645 718 leader_election.cc:304] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:19.925176 3199 raft_consensus.cc:2804] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.925554 3199 raft_consensus.cc:697] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:19.925874 3199 consensus_queue.cc:237] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.916499 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8e6019406845ebb29051432a96dfd1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.926700 3363 leader_election.cc:290] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.920786 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a8e6019406845ebb29051432a96dfd1" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:19.927186 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efb07cb60f64dfead739a3ca1343df9" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:19.927230 1052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efb07cb60f64dfead739a3ca1343df9" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:19.927332 912 raft_consensus.cc:3060] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
W20260505 07:54:19.927778 717 leader_election.cc:343] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.923954 778 raft_consensus.cc:2468] T 01431ad112e8445487823efe8feb0dbe P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:19.930532 912 raft_consensus.cc:2468] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:19.866490 1049 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "68e608ac01ef4a88906e585bd7ee25ac" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.867287 1055 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "73042a7ff5fb43f7a781b4f98f4f4020" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:19.916712 2447 ts_tablet_manager.cc:1929] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.933478 2447 log.cc:1199] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/141665fcd59048bf926acd0287f1853e
I20260505 07:54:19.933743 1048 raft_consensus.cc:2468] T 68e608ac01ef4a88906e585bd7ee25ac P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:19.934649 2447 ts_tablet_manager.cc:1950] T 141665fcd59048bf926acd0287f1853e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.934980 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 210) took 10168 ms. Trace:
I20260505 07:54:19.935057 2447 rpcz_store.cc:276] 0505 07:54:09.766465 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.767020 (+ 555us) service_pool.cc:224] Handling call
0505 07:54:19.807816 (+10040796us) ts_tablet_manager.cc:1151] Deleting tablet 141665fcd59048bf926acd0287f1853e
0505 07:54:19.807825 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.855826 (+ 48001us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.916646 (+ 60820us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.934963 (+ 18317us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10040643}
I20260505 07:54:19.935457 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 141665fcd59048bf926acd0287f1853e (table default.Table_0ec687e90aea4825b28b36eda22678fe [id=c551b1aa1f2e4eaab4ce7c6727c05465]) successfully deleted
I20260505 07:54:19.926652 782 raft_consensus.cc:2468] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 0.
W20260505 07:54:19.934249 856 leader_election.cc:343] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:19.938678 2447 tablet_replica.cc:333] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.938653 856 leader_election.cc:304] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.939370 3361 raft_consensus.cc:2749] T 73042a7ff5fb43f7a781b4f98f4f4020 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:19.910274 2276 rpcz_store.cc:276] 0505 07:54:10.942097 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.942143 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:19.910112 (+8967969us) ts_tablet_manager.cc:1151] Deleting tablet 06904a54b1c5463da31af0bc00fb606f
0505 07:54:19.910122 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.910187 (+ 65us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8967786}
I20260505 07:54:19.939553 1047 raft_consensus.cc:2468] T 09019fdf60374b11bf3427f21a55b5e0 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:19.939744 2268 raft_consensus.cc:2272] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:19.940088 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 06904a54b1c5463da31af0bc00fb606f because the tablet was not found. No further retry: Not found: Tablet not found: 06904a54b1c5463da31af0bc00fb606f
I20260505 07:54:19.940500 2268 ts_tablet_manager.cc:1916] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.940537 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 212) took 8998 ms. Trace:
I20260505 07:54:19.940702 2276 rpcz_store.cc:276] 0505 07:54:10.942435 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.942480 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:19.940443 (+8997963us) ts_tablet_manager.cc:1151] Deleting tablet 26f3544e74d941d286a2a0317b941ab6
0505 07:54:19.940453 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.940521 (+ 68us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8997837}
I20260505 07:54:19.941195 718 leader_election.cc:304] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.941700 3199 raft_consensus.cc:2804] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.941861 3199 raft_consensus.cc:697] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:19.942055 1050 raft_consensus.cc:2468] T 4a8e6019406845ebb29051432a96dfd1 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 1.
I20260505 07:54:19.942134 3199 consensus_queue.cc:237] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.942436 856 leader_election.cc:304] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:19.943647 3361 raft_consensus.cc:2804] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:19.943931 3361 raft_consensus.cc:697] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:19.944320 3361 consensus_queue.cc:237] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:19.944839 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 26f3544e74d941d286a2a0317b941ab6 because the tablet was not found. No further retry: Not found: Tablet not found: 26f3544e74d941d286a2a0317b941ab6
I20260505 07:54:19.945526 3363 raft_consensus.cc:493] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d2799f5c6b4f4336b976ca9bbb458533)
I20260505 07:54:19.945681 3363 raft_consensus.cc:515] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.946336 3363 leader_election.cc:290] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:19.945951 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 213) took 9003 ms. Trace:
I20260505 07:54:19.946688 2276 rpcz_store.cc:276] 0505 07:54:10.942721 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.942761 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:19.945772 (+9003011us) ts_tablet_manager.cc:1151] Deleting tablet 4662d4a3d88d4c2883a35048d6e11eb9
0505 07:54:19.945786 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.945931 (+ 145us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9002882}
W20260505 07:54:19.947652 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 4662d4a3d88d4c2883a35048d6e11eb9 because the tablet was not found. No further retry: Not found: Tablet not found: 4662d4a3d88d4c2883a35048d6e11eb9
I20260505 07:54:19.947994 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 210) took 9006 ms. Trace:
I20260505 07:54:19.948285 2276 rpcz_store.cc:276] 0505 07:54:10.942000 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.943314 (+ 1314us) service_pool.cc:224] Handling call
0505 07:54:19.947911 (+9004597us) ts_tablet_manager.cc:1151] Deleting tablet 8b406c4c6ba741bc85eef54c6f576459
0505 07:54:19.947921 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.947981 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9004402}
W20260505 07:54:19.948853 717 leader_election.cc:343] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: b256d072933b4566b5587224274deb66
I20260505 07:54:19.949002 2447 raft_consensus.cc:2243] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.949892 2447 raft_consensus.cc:2272] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.950387 2447 ts_tablet_manager.cc:1916] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:19.951516 718 leader_election.cc:343] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: b256d072933b4566b5587224274deb66
I20260505 07:54:19.951634 718 leader_election.cc:304] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:19.951915 3364 raft_consensus.cc:2749] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260505 07:54:19.955529 1181 tablet_bootstrap.cc:492] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.955706 1181 ts_tablet_manager.cc:1403] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.157s user 0.004s sys 0.005s
I20260505 07:54:19.956383 1181 raft_consensus.cc:359] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.956537 1181 raft_consensus.cc:385] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.956588 1181 raft_consensus.cc:740] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.956761 1181 consensus_queue.cc:260] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.957342 1181 ts_tablet_manager.cc:1434] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:19.957610 1181 tablet_bootstrap.cc:492] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:19.958290 1176 tablet_bootstrap.cc:492] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:19.958703 1176 ts_tablet_manager.cc:1403] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.071s user 0.002s sys 0.001s
I20260505 07:54:19.959141 1181 tablet_bootstrap.cc:654] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.959717 1176 raft_consensus.cc:359] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.959930 1176 raft_consensus.cc:385] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.960045 1176 raft_consensus.cc:740] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.960348 1176 consensus_queue.cc:260] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.964375 1172 tablet_bootstrap.cc:492] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:19.964661 1172 ts_tablet_manager.cc:1403] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.157s user 0.009s sys 0.004s
I20260505 07:54:19.965236 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 215) took 9019 ms. Trace:
I20260505 07:54:19.965409 3378 raft_consensus.cc:493] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 23c57a2c927c4c54bccecfeb3a775311)
I20260505 07:54:19.965373 2276 rpcz_store.cc:276] 0505 07:54:10.945694 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.945735 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:19.965143 (+9019408us) ts_tablet_manager.cc:1151] Deleting tablet 43e4c92b7c3140fdb2d6a290aaf0ad55
0505 07:54:19.965154 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.965220 (+ 66us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9019252}
I20260505 07:54:19.965718 3378 raft_consensus.cc:515] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "379cdfdecc3341ea9d7d9dc09274338d" member_type: VOTER last_known_addr { host: "127.0.105.62" port: 33621 } } peers { permanent_uuid: "23c57a2c927c4c54bccecfeb3a775311" member_type: VOTER last_known_addr { host: "127.0.105.61" port: 33565 } } peers { permanent_uuid: "28d3eda04f9c47ae976d19c218b6dcc5" member_type: VOTER last_known_addr { host: "127.0.105.60" port: 35401 } }
I20260505 07:54:19.965850 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 214) took 9022 ms. Trace:
I20260505 07:54:19.965914 2276 rpcz_store.cc:276] 0505 07:54:10.943734 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.947647 (+ 3913us) service_pool.cc:224] Handling call
0505 07:54:19.965777 (+9018130us) ts_tablet_manager.cc:1151] Deleting tablet a7fe9c7199cc4039ae877dc3d4aab17e
0505 07:54:19.965786 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.965836 (+ 50us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9017905}
I20260505 07:54:19.965794 1172 raft_consensus.cc:359] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.966132 1172 raft_consensus.cc:385] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.966214 1172 raft_consensus.cc:740] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.966334 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 216) took 8965 ms. Trace:
I20260505 07:54:19.966425 2276 rpcz_store.cc:276] 0505 07:54:11.001179 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.001246 (+ 67us) service_pool.cc:224] Handling call
0505 07:54:19.966235 (+8964989us) ts_tablet_manager.cc:1151] Deleting tablet 880e326cbea74233a0ec5abb31f5b0d7
0505 07:54:19.966245 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.966320 (+ 75us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8959742}
I20260505 07:54:19.966506 1172 consensus_queue.cc:260] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.966758 2276 tablet_replica.cc:333] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:19.966954 2276 raft_consensus.cc:2243] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:19.967103 2276 raft_consensus.cc:2272] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:19.967824 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 8b406c4c6ba741bc85eef54c6f576459 because the tablet was not found. No further retry: Not found: Tablet not found: 8b406c4c6ba741bc85eef54c6f576459
W20260505 07:54:19.968120 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 because the tablet was not found. No further retry: Not found: Tablet not found: 43e4c92b7c3140fdb2d6a290aaf0ad55
W20260505 07:54:19.968354 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a7fe9c7199cc4039ae877dc3d4aab17e because the tablet was not found. No further retry: Not found: Tablet not found: a7fe9c7199cc4039ae877dc3d4aab17e
W20260505 07:54:19.968591 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 880e326cbea74233a0ec5abb31f5b0d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 880e326cbea74233a0ec5abb31f5b0d7 already in progress: creating tablet
I20260505 07:54:19.968521 1172 ts_tablet_manager.cc:1434] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:19.970168 3202 raft_consensus.cc:493] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:19.970350 3202 raft_consensus.cc:515] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.970626 1172 tablet_bootstrap.cc:492] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:19.970952 1378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "379cdfdecc3341ea9d7d9dc09274338d" candidate_term: 3 candidate_status { last_received { term: 2 index: 882 } } ignore_live_leader: false dest_uuid: "23c57a2c927c4c54bccecfeb3a775311" is_pre_election: true
I20260505 07:54:19.971450 3202 leader_election.cc:290] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:19.972321 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d896e290167b4d0081e8e8b831c8ae2a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:19.972918 918 raft_consensus.cc:2468] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.973372 991 leader_election.cc:304] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:19.973752 3202 raft_consensus.cc:2804] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:19.973891 3202 raft_consensus.cc:493] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:19.974038 3202 raft_consensus.cc:3060] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:19.979264 3378 leader_election.cc:290] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 23c57a2c927c4c54bccecfeb3a775311 (127.0.105.61:33565), 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401)
I20260505 07:54:19.984186 1172 tablet_bootstrap.cc:654] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:19.989259 1176 ts_tablet_manager.cc:1434] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.030s user 0.001s sys 0.000s
I20260505 07:54:19.972064 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d896e290167b4d0081e8e8b831c8ae2a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:19.989742 782 raft_consensus.cc:2468] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:19.972832 2447 ts_tablet_manager.cc:1929] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.989975 2447 log.cc:1199] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/d8c9eaaef250461ca7a37d0cf42b07ee
I20260505 07:54:19.990506 2447 ts_tablet_manager.cc:1950] T d8c9eaaef250461ca7a37d0cf42b07ee P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.991130 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 213) took 10224 ms. Trace:
I20260505 07:54:19.991205 2447 rpcz_store.cc:276] 0505 07:54:09.766999 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.767194 (+ 195us) service_pool.cc:224] Handling call
0505 07:54:19.938635 (+10171441us) ts_tablet_manager.cc:1151] Deleting tablet d8c9eaaef250461ca7a37d0cf42b07ee
0505 07:54:19.938644 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.964950 (+ 26306us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.972783 (+ 7833us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.991117 (+ 18334us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10171333}
I20260505 07:54:19.991793 2447 tablet_replica.cc:333] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.991758 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet d8c9eaaef250461ca7a37d0cf42b07ee (table default.Table_92db3a6ebf824942966e677a2cd23ce1 [id=4890bd76a6f04392848038377244273d]) successfully deleted
I20260505 07:54:19.991971 2447 raft_consensus.cc:2243] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:19.992115 1181 tablet_bootstrap.cc:492] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:19.992231 1181 ts_tablet_manager.cc:1403] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.002s
I20260505 07:54:19.992281 2447 raft_consensus.cc:2272] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:19.992767 2447 ts_tablet_manager.cc:1916] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.992867 1181 raft_consensus.cc:359] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.993019 1181 raft_consensus.cc:385] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:19.993080 1181 raft_consensus.cc:740] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:19.993266 1181 consensus_queue.cc:260] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:19.994566 3396 tablet.cc:1724] T de1a667b14fe494c8ee4be0f47a340be P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:19.998379 2447 ts_tablet_manager.cc:1929] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:19.998466 2447 log.cc:1199] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/4258df74ea0841449846cdf1f6e00b9b
I20260505 07:54:19.998893 2447 ts_tablet_manager.cc:1950] T 4258df74ea0841449846cdf1f6e00b9b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:19.999457 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 214) took 10212 ms. Trace:
I20260505 07:54:19.999544 2447 rpcz_store.cc:276] 0505 07:54:09.786960 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.788253 (+ 1293us) service_pool.cc:224] Handling call
0505 07:54:19.991750 (+10203497us) ts_tablet_manager.cc:1151] Deleting tablet 4258df74ea0841449846cdf1f6e00b9b
0505 07:54:19.991760 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.995773 (+ 4013us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.998340 (+ 2567us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:19.999147 (+ 807us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10201673}
I20260505 07:54:19.999970 2447 tablet_replica.cc:333] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:19.999936 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 4258df74ea0841449846cdf1f6e00b9b (table default.Table_b3b9364eea524724b176de87f62eb49e [id=1fd77c8cfe9d40dab5da490774294796]) successfully deleted
I20260505 07:54:20.000162 2447 raft_consensus.cc:2243] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.000476 2447 raft_consensus.cc:2272] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.000982 2447 ts_tablet_manager.cc:1916] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.001693 1181 ts_tablet_manager.cc:1434] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260505 07:54:20.002488 1181 tablet_bootstrap.cc:492] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.006758 2447 ts_tablet_manager.cc:1929] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.006843 2447 log.cc:1199] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/b5a0cbf9c3d74cea92920189da332c1c
I20260505 07:54:20.006942 3296 raft_consensus.cc:493] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.007140 2447 ts_tablet_manager.cc:1950] T b5a0cbf9c3d74cea92920189da332c1c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.007071 3296 raft_consensus.cc:515] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.007416 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 215) took 10218 ms. Trace:
I20260505 07:54:20.007476 2447 rpcz_store.cc:276] 0505 07:54:09.789947 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.790347 (+ 400us) service_pool.cc:224] Handling call
0505 07:54:19.999928 (+10209581us) ts_tablet_manager.cc:1151] Deleting tablet b5a0cbf9c3d74cea92920189da332c1c
0505 07:54:19.999937 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.003755 (+ 3818us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.006690 (+ 2935us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.007402 (+ 712us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10209452}
I20260505 07:54:20.007611 3296 leader_election.cc:290] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.007866 2447 tablet_replica.cc:333] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.007881 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet b5a0cbf9c3d74cea92920189da332c1c (table default.Table_e2720de78a7742d5829d7db2ef803f0d [id=f99b727e2aa54591baf6f109c07c2677]) successfully deleted
I20260505 07:54:20.008020 2447 raft_consensus.cc:2243] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.008288 2447 raft_consensus.cc:2272] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.008576 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2977a0a646448249d9fbe46f970d63b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.008750 2447 ts_tablet_manager.cc:1916] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:20.009121 856 leader_election.cc:343] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.010752 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2977a0a646448249d9fbe46f970d63b" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:20.011261 854 leader_election.cc:343] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.011365 854 leader_election.cc:304] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.011912 3296 raft_consensus.cc:2749] T f2977a0a646448249d9fbe46f970d63b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.014330 1176 tablet_bootstrap.cc:492] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:20.014503 2447 ts_tablet_manager.cc:1929] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.014597 2447 log.cc:1199] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/d3d11bd19f074bbab101448b45426b7a
I20260505 07:54:20.014916 2447 ts_tablet_manager.cc:1950] T d3d11bd19f074bbab101448b45426b7a P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.015195 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 216) took 10224 ms. Trace:
I20260505 07:54:20.015268 2447 rpcz_store.cc:276] 0505 07:54:09.792043 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.792088 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:20.007826 (+10215738us) ts_tablet_manager.cc:1151] Deleting tablet d3d11bd19f074bbab101448b45426b7a
0505 07:54:20.007835 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.011464 (+ 3629us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.014462 (+ 2998us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.015183 (+ 721us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10215600}
I20260505 07:54:20.015648 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet d3d11bd19f074bbab101448b45426b7a (table default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871]) successfully deleted
I20260505 07:54:20.015731 2447 tablet_replica.cc:333] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.015893 2447 raft_consensus.cc:2243] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.016028 2447 raft_consensus.cc:2272] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.016163 1176 tablet_bootstrap.cc:654] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.016456 2447 ts_tablet_manager.cc:1916] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.004207 1181 tablet_bootstrap.cc:654] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.018654 3397 tablet.cc:1724] T de1a667b14fe494c8ee4be0f47a340be P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:20.018785 1172 tablet_bootstrap.cc:492] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.018901 1172 ts_tablet_manager.cc:1403] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.048s user 0.002s sys 0.001s
I20260505 07:54:20.019696 1172 raft_consensus.cc:359] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.019869 1172 raft_consensus.cc:385] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.019943 1172 raft_consensus.cc:740] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.020287 1172 consensus_queue.cc:260] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.020718 1172 ts_tablet_manager.cc:1434] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:20.021875 1172 tablet_bootstrap.cc:492] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:20.023296 1172 tablet_bootstrap.cc:654] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.019708 2276 ts_tablet_manager.cc:1916] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.027400 1172 tablet_bootstrap.cc:492] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.027539 1172 ts_tablet_manager.cc:1403] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260505 07:54:20.028728 1172 raft_consensus.cc:359] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.028904 1172 raft_consensus.cc:385] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.028980 1172 raft_consensus.cc:740] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.029184 1172 consensus_queue.cc:260] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.029608 1172 ts_tablet_manager.cc:1434] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:20.029903 1172 tablet_bootstrap.cc:492] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:20.031957 1172 tablet_bootstrap.cc:654] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.022682 2447 ts_tablet_manager.cc:1929] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.032320 2447 log.cc:1199] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/7001482d90364dbdb8b9c5bfb98e1cd4
I20260505 07:54:20.032775 2447 ts_tablet_manager.cc:1950] T 7001482d90364dbdb8b9c5bfb98e1cd4 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.032966 3361 raft_consensus.cc:493] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.033139 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 217) took 10239 ms. Trace:
I20260505 07:54:20.033144 3361 raft_consensus.cc:515] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.033223 2447 rpcz_store.cc:276] 0505 07:54:09.807224 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.807282 (+ 58us) service_pool.cc:224] Handling call
0505 07:54:20.015676 (+10208394us) ts_tablet_manager.cc:1151] Deleting tablet 7001482d90364dbdb8b9c5bfb98e1cd4
0505 07:54:20.015685 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.019721 (+ 4036us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.022633 (+ 2912us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.033118 (+ 10485us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10208224}
I20260505 07:54:20.033670 2447 tablet_replica.cc:333] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.033627 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 7001482d90364dbdb8b9c5bfb98e1cd4 (table default.Table_304a4522acf84575bc1b2bdbf22a7ac4 [id=438fe94f1aab49919a5aae9ccab8f491]) successfully deleted
I20260505 07:54:20.033841 2447 raft_consensus.cc:2243] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.033982 2447 raft_consensus.cc:2272] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.034472 2447 ts_tablet_manager.cc:1916] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.034653 3361 leader_election.cc:290] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:20.034934 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f31343e5674c0eabb90da680bc34a4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:20.035539 856 leader_election.cc:343] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.040553 2447 ts_tablet_manager.cc:1929] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.040661 2276 ts_tablet_manager.cc:1929] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.040818 2276 log.cc:1199] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/f842d8ac6b2142ffa69945c6087be34b
I20260505 07:54:20.041360 2276 ts_tablet_manager.cc:1950] T f842d8ac6b2142ffa69945c6087be34b P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.041613 2447 log.cc:1199] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/02412333633a4cdebe95a144fa6c363c
I20260505 07:54:20.042418 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 217) took 8998 ms. Trace:
I20260505 07:54:20.042551 2276 rpcz_store.cc:276] 0505 07:54:11.044095 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.044155 (+ 60us) service_pool.cc:224] Handling call
0505 07:54:19.966717 (+8922562us) ts_tablet_manager.cc:1151] Deleting tablet f842d8ac6b2142ffa69945c6087be34b
0505 07:54:19.966725 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.025904 (+ 59179us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.040603 (+ 14699us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.042363 (+ 1760us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8922378}
I20260505 07:54:20.043103 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]) successfully deleted
I20260505 07:54:20.043556 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0f31343e5674c0eabb90da680bc34a4" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.044505 2268 ts_tablet_manager.cc:1929] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260505 07:54:20.044549 854 leader_election.cc:343] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.044641 2268 log.cc:1199] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/3075e87e078f4d2aa1d329d18902354b
I20260505 07:54:20.044708 854 leader_election.cc:304] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.044976 2268 ts_tablet_manager.cc:1950] T 3075e87e078f4d2aa1d329d18902354b P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:20.045357 3361 raft_consensus.cc:2749] T c0f31343e5674c0eabb90da680bc34a4 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.045678 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 320) took 5359 ms. Trace:
I20260505 07:54:20.045817 2447 ts_tablet_manager.cc:1950] T 02412333633a4cdebe95a144fa6c363c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.045784 2268 rpcz_store.cc:276] 0505 07:54:14.686311 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.686346 (+ 35us) service_pool.cc:224] Handling call
0505 07:54:19.531270 (+4844924us) ts_tablet_manager.cc:1151] Deleting tablet 3075e87e078f4d2aa1d329d18902354b
0505 07:54:19.531278 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:19.943404 (+412126us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.044451 (+101047us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.045660 (+ 1209us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4844774}
I20260505 07:54:20.024732 3177 raft_consensus.cc:697] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:20.046211 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 3075e87e078f4d2aa1d329d18902354b (table default.Table_c1cf7717be07400aacdab563b4255450 [id=4d79fded98df46cc884cabdc3fba5d22]) successfully deleted
I20260505 07:54:20.046686 3177 consensus_queue.cc:237] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:20.047545 1349 catalog_manager.cc:5931] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 260a8d41de464e429491ada4bfe2c761 consensus state has no leader
I20260505 07:54:20.047756 1349 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:46452 (request call id 272) took 1449 ms. Trace:
I20260505 07:54:20.047847 1349 rpcz_store.cc:276] 0505 07:54:18.598448 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:19.875429 (+1276981us) service_pool.cc:224] Handling call
0505 07:54:19.875887 (+ 458us) catalog_manager.cc:2393] Looking up, locking, and authorizing table
0505 07:54:19.876158 (+ 271us) catalog_manager.cc:3725] Apply alter schema
0505 07:54:19.876406 (+ 248us) catalog_manager.cc:3757] Acquired catalog manager lock
0505 07:54:19.876503 (+ 97us) catalog_manager.cc:3915] Updating metadata on disk
0505 07:54:20.047260 (+170757us) catalog_manager.cc:3946] Committing alterations to in-memory state
0505 07:54:20.047705 (+ 445us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:19.885124 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:19.885241 (+ 117us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:19.885249 (+ 8us) write_op.cc:435] Acquired schema lock
0505 07:54:19.885251 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:19.885373 (+ 122us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:19.885389 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:19.885394 (+ 5us) tablet.cc:685] Acquiring locks for 1 operations
0505 07:54:19.885460 (+ 66us) tablet.cc:701] Row locks acquired
0505 07:54:19.885463 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:19.885496 (+ 33us) write_op.cc:270] Start()
0505 07:54:19.885530 (+ 34us) write_op.cc:276] Timestamp: P: 1777967659885436 usec, L: 0
0505 07:54:19.885534 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:19.885723 (+ 189us) log.cc:844] Serialized 405 byte log entry
0505 07:54:19.980074 (+ 94351us) op_driver.cc:464] REPLICATION: finished
0505 07:54:20.046698 (+ 66624us) write_op.cc:301] APPLY: starting
0505 07:54:20.046720 (+ 22us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:20.046772 (+ 52us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:20.046811 (+ 39us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:20.046876 (+ 65us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:20.046912 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:20.046920 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:20.047038 (+ 118us) log.cc:844] Serialized 23 byte log entry
0505 07:54:20.047144 (+ 106us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.047164 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:20.047174 (+ 10us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":66521,"apply.run_cpu_time_us":568,"apply.run_wall_time_us":568,"num_ops":1,"prepare.queue_time_us":7917,"prepare.run_cpu_time_us":3327,"prepare.run_wall_time_us":38397,"raft.queue_time_us":14605,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":176,"replication_time_us":94489,"thread_start_us":42779,"threads_started":4}]]}
I20260505 07:54:20.048628 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 218) took 10241 ms. Trace:
I20260505 07:54:20.048697 2447 rpcz_store.cc:276] 0505 07:54:09.807350 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.807489 (+ 139us) service_pool.cc:224] Handling call
0505 07:54:20.033623 (+10226134us) ts_tablet_manager.cc:1151] Deleting tablet 02412333633a4cdebe95a144fa6c363c
0505 07:54:20.033634 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.037310 (+ 3676us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.040503 (+ 3193us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.048610 (+ 8107us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10225944}
I20260505 07:54:20.049607 2447 tablet_replica.cc:333] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.049785 2447 raft_consensus.cc:2243] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.049912 2447 raft_consensus.cc:2272] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.050406 2447 ts_tablet_manager.cc:1916] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:19.977293 3202 raft_consensus.cc:515] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.053004 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d896e290167b4d0081e8e8b831c8ae2a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:20.053246 912 raft_consensus.cc:3060] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.054988 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d896e290167b4d0081e8e8b831c8ae2a" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:20.055262 778 raft_consensus.cc:3060] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.058759 3403 tablet.cc:1724] T de1a667b14fe494c8ee4be0f47a340be P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260505 07:54:20.058841 778 raft_consensus.cc:2468] T d896e290167b4d0081e8e8b831c8ae2a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:20.059303 988 leader_election.cc:304] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:20.059579 3203 raft_consensus.cc:2804] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:20.059783 3203 raft_consensus.cc:697] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:20.059784 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 02412333633a4cdebe95a144fa6c363c (table default.Table_0654ae2daf60435697eeca15ba0c19d7 [id=13d54930c52f4e9eae01bc80a3f1d565]) successfully deleted
I20260505 07:54:20.059895 1181 tablet_bootstrap.cc:492] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.060001 3203 consensus_queue.cc:237] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.061472 1176 tablet_bootstrap.cc:492] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:20.061583 1176 ts_tablet_manager.cc:1403] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.047s user 0.002s sys 0.000s
I20260505 07:54:20.062296 1176 raft_consensus.cc:359] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.062480 1176 raft_consensus.cc:385] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.062541 1176 raft_consensus.cc:740] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.060007 1181 ts_tablet_manager.cc:1403] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.058s user 0.003s sys 0.000s
I20260505 07:54:20.062741 1176 consensus_queue.cc:260] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.063624 1181 raft_consensus.cc:359] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.063766 1181 raft_consensus.cc:385] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.063827 1181 raft_consensus.cc:740] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.064028 1181 consensus_queue.cc:260] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.064704 3202 leader_election.cc:290] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:20.065176 1181 ts_tablet_manager.cc:1434] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:20.067456 912 raft_consensus.cc:2468] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:20.070864 1176 ts_tablet_manager.cc:1434] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260505 07:54:20.071303 1181 tablet_bootstrap.cc:492] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.071748 1176 tablet_bootstrap.cc:492] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:20.072994 1181 tablet_bootstrap.cc:654] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.074070 2276 tablet_replica.cc:333] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.074412 2276 raft_consensus.cc:2243] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.074941 2276 pending_rounds.cc:70] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:54:20.075031 2276 pending_rounds.cc:77] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 2 index: 1 } timestamp: 7282555533541040128 op_type: NO_OP noop_request { }
I20260505 07:54:20.075477 2276 raft_consensus.cc:2889] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:20.075606 2276 raft_consensus.cc:2272] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.076159 1176 tablet_bootstrap.cc:654] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:20.078717 444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:54:20.079998 2447 ts_tablet_manager.cc:1929] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.080102 2447 log.cc:1199] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/46274a44a06b4306b5dac43c66c85a80
I20260505 07:54:20.080446 2447 ts_tablet_manager.cc:1950] T 46274a44a06b4306b5dac43c66c85a80 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.080660 1357 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 274) took 1459 ms. Trace:
I20260505 07:54:20.080706 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 219) took 10273 ms. Trace:
I20260505 07:54:20.080767 2447 rpcz_store.cc:276] 0505 07:54:09.807650 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.807703 (+ 53us) service_pool.cc:224] Handling call
0505 07:54:20.049563 (+10241860us) ts_tablet_manager.cc:1151] Deleting tablet 46274a44a06b4306b5dac43c66c85a80
0505 07:54:20.049573 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.069522 (+ 19949us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.079953 (+ 10431us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.080693 (+ 740us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10241747}
I20260505 07:54:20.080785 1357 rpcz_store.cc:276] 0505 07:54:18.621424 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:19.890340 (+1268916us) service_pool.cc:224] Handling call
0505 07:54:19.891239 (+ 899us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:19.892370 (+ 1131us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:20.080412 (+188042us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:20.080520 (+ 108us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:20.080621 (+ 101us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:19.923553 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:19.923726 (+ 173us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:19.923736 (+ 10us) write_op.cc:435] Acquired schema lock
0505 07:54:19.923740 (+ 4us) tablet.cc:662] Decoding operations
0505 07:54:19.923861 (+ 121us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:19.923877 (+ 16us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:19.923883 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:19.923999 (+ 116us) tablet.cc:701] Row locks acquired
0505 07:54:19.924003 (+ 4us) write_op.cc:260] PREPARE: finished
0505 07:54:19.924045 (+ 42us) write_op.cc:270] Start()
0505 07:54:19.924089 (+ 44us) write_op.cc:276] Timestamp: P: 1777967659923978 usec, L: 0
0505 07:54:19.924095 (+ 6us) op_driver.cc:348] REPLICATION: starting
0505 07:54:19.924300 (+ 205us) log.cc:844] Serialized 433 byte log entry
0505 07:54:20.079430 (+155130us) op_driver.cc:464] REPLICATION: finished
0505 07:54:20.079545 (+ 115us) write_op.cc:301] APPLY: starting
0505 07:54:20.079562 (+ 17us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:20.079641 (+ 79us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:20.079666 (+ 25us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:20.079797 (+ 131us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:20.079844 (+ 47us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:20.079855 (+ 11us) write_op.cc:312] APPLY: finished
0505 07:54:20.079935 (+ 80us) log.cc:844] Serialized 29 byte log entry
0505 07:54:20.080284 (+ 349us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.080307 (+ 23us) write_op.cc:454] Released schema lock
0505 07:54:20.080317 (+ 10us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":71,"num_ops":2,"prepare.queue_time_us":30744,"prepare.run_cpu_time_us":909,"prepare.run_wall_time_us":1470,"replication_time_us":155306}]]}
I20260505 07:54:20.081961 2447 tablet_replica.cc:333] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.082165 2447 raft_consensus.cc:2243] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.082297 2447 raft_consensus.cc:2272] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.082754 2447 ts_tablet_manager.cc:1916] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.083850 3361 raft_consensus.cc:493] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.083979 3361 raft_consensus.cc:515] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.084546 3361 leader_election.cc:290] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.085135 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f25c3e1f7e442cba8c837b568c7b53f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:20.085645 856 leader_election.cc:343] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.086066 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f25c3e1f7e442cba8c837b568c7b53f" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:20.086951 854 leader_election.cc:343] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.087074 854 leader_election.cc:304] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.087401 3361 raft_consensus.cc:2749] T 6f25c3e1f7e442cba8c837b568c7b53f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.087699 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 46274a44a06b4306b5dac43c66c85a80 (table default.Table_614ea89c84d342f0a1c4912e8e77fe47 [id=f1230f25280e45269ffbd95bbe8dbbfc]) successfully deleted
I20260505 07:54:20.089835 3199 raft_consensus.cc:493] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.089954 3199 raft_consensus.cc:515] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.090185 2276 ts_tablet_manager.cc:1916] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.090478 3199 leader_election.cc:290] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.091068 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac029d9a3c540f58f393e79bf7be30e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.091161 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac029d9a3c540f58f393e79bf7be30e" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.091554 718 leader_election.cc:304] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:20.091913 3199 raft_consensus.cc:2749] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.092939 3199 raft_consensus.cc:493] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.093040 3199 raft_consensus.cc:515] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.093531 3199 leader_election.cc:290] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.093703 2447 ts_tablet_manager.cc:1929] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.093822 2447 log.cc:1199] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/e29ba55a846548679cb3b86305d0d6ed
I20260505 07:54:20.093765 3296 consensus_queue.cc:1048] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.094195 2447 ts_tablet_manager.cc:1950] T e29ba55a846548679cb3b86305d0d6ed P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.094259 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a3cd77808c4e858ea2ef9ed2a28c38" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.094398 1051 raft_consensus.cc:2468] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:20.094498 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 220) took 10286 ms. Trace:
I20260505 07:54:20.094583 2447 rpcz_store.cc:276] 0505 07:54:09.807791 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.807833 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:20.081915 (+10274082us) ts_tablet_manager.cc:1151] Deleting tablet e29ba55a846548679cb3b86305d0d6ed
0505 07:54:20.081924 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.085506 (+ 3582us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.093631 (+ 8125us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.094463 (+ 832us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10273993}
I20260505 07:54:20.094762 717 leader_election.cc:304] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:20.094883 2447 tablet_replica.cc:333] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.095101 3199 raft_consensus.cc:2804] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:20.095220 3199 raft_consensus.cc:493] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:20.095304 3199 raft_consensus.cc:3060] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.095511 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet e29ba55a846548679cb3b86305d0d6ed (table default.Table_fd75ee1f117344169ccafd58dc18408f [id=5cd6a3d7d65d4a43943fa891ca480d7e]) successfully deleted
I20260505 07:54:20.098034 3199 raft_consensus.cc:515] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.098971 3199 leader_election.cc:290] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.100562 1181 tablet_bootstrap.cc:492] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.100632 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a3cd77808c4e858ea2ef9ed2a28c38" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:20.100706 1181 ts_tablet_manager.cc:1403] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.030s user 0.003s sys 0.000s
I20260505 07:54:20.100708 1355 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 275) took 1478 ms. Trace:
I20260505 07:54:20.100821 907 raft_consensus.cc:3060] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.101361 1181 raft_consensus.cc:359] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.100842 1355 rpcz_store.cc:276] 0505 07:54:18.622195 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:19.908779 (+1286584us) service_pool.cc:224] Handling call
0505 07:54:19.909756 (+ 977us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:19.910241 (+ 485us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:20.100404 (+190163us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:20.100536 (+ 132us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:20.100648 (+ 112us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:19.925054 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:19.925172 (+ 118us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:19.925177 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:19.925179 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:19.925268 (+ 89us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:19.925278 (+ 10us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:19.925283 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:19.925358 (+ 75us) tablet.cc:701] Row locks acquired
0505 07:54:19.925360 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:19.925389 (+ 29us) write_op.cc:270] Start()
0505 07:54:19.925419 (+ 30us) write_op.cc:276] Timestamp: P: 1777967659925326 usec, L: 0
0505 07:54:19.925422 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:19.925572 (+ 150us) log.cc:844] Serialized 433 byte log entry
0505 07:54:20.099549 (+173977us) op_driver.cc:464] REPLICATION: finished
0505 07:54:20.099659 (+ 110us) write_op.cc:301] APPLY: starting
0505 07:54:20.099678 (+ 19us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:20.099729 (+ 51us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:20.099761 (+ 32us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:20.099909 (+ 148us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:20.099953 (+ 44us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:20.099965 (+ 12us) write_op.cc:312] APPLY: finished
0505 07:54:20.100059 (+ 94us) log.cc:844] Serialized 29 byte log entry
0505 07:54:20.100296 (+ 237us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.100316 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:20.100322 (+ 6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":57,"num_ops":2,"prepare.queue_time_us":14342,"prepare.run_cpu_time_us":584,"prepare.run_wall_time_us":875,"replication_time_us":174094}]]}
I20260505 07:54:20.101495 1181 raft_consensus.cc:385] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.101538 1181 raft_consensus.cc:740] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.101734 1181 consensus_queue.cc:260] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.102183 1181 ts_tablet_manager.cc:1434] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260505 07:54:20.102417 1181 tablet_bootstrap.cc:492] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.095096 2447 raft_consensus.cc:2243] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.102991 2447 raft_consensus.cc:2272] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.103278 907 raft_consensus.cc:2468] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:20.103543 2447 ts_tablet_manager.cc:1916] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.104079 1181 tablet_bootstrap.cc:654] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.104789 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a3cd77808c4e858ea2ef9ed2a28c38" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:20.104933 1050 raft_consensus.cc:3060] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.107270 1050 raft_consensus.cc:2468] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:20.107904 718 leader_election.cc:304] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:20.108269 3363 raft_consensus.cc:2804] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:20.108392 3363 raft_consensus.cc:697] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:20.108628 3363 consensus_queue.cc:237] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.111249 3144 raft_consensus.cc:493] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.111423 3144 raft_consensus.cc:515] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.112166 3144 leader_election.cc:290] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:20.112505 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84faf2e3d91e4cc394a3411c5b6c0bc7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:20.112910 854 leader_election.cc:343] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.112994 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84faf2e3d91e4cc394a3411c5b6c0bc7" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:20.113533 856 leader_election.cc:343] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.113677 856 leader_election.cc:304] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.114270 3361 raft_consensus.cc:2749] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.114925 3199 raft_consensus.cc:493] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.115052 3199 raft_consensus.cc:515] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.115509 3199 leader_election.cc:290] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.116257 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172ca66c49354f549fa55fe63abf9970" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.116451 918 raft_consensus.cc:2393] T 172ca66c49354f549fa55fe63abf9970 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:20.116804 2447 ts_tablet_manager.cc:1929] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.116933 2447 log.cc:1199] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/3075e87e078f4d2aa1d329d18902354b
I20260505 07:54:20.117340 2447 ts_tablet_manager.cc:1950] T 3075e87e078f4d2aa1d329d18902354b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.117645 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 221) took 10309 ms. Trace:
I20260505 07:54:20.117755 3364 raft_consensus.cc:493] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.117913 3364 raft_consensus.cc:515] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.118367 3364 leader_election.cc:290] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
W20260505 07:54:20.119324 717 leader_election.cc:343] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 727cacd242844455915bd9963d2db19f
I20260505 07:54:20.117743 2447 rpcz_store.cc:276] 0505 07:54:09.807905 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.807937 (+ 32us) service_pool.cc:224] Handling call
0505 07:54:20.094814 (+10286877us) ts_tablet_manager.cc:1151] Deleting tablet 3075e87e078f4d2aa1d329d18902354b
0505 07:54:20.094824 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.108217 (+ 13393us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.116731 (+ 8514us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.117602 (+ 871us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10286789}
W20260505 07:54:20.120420 718 leader_election.cc:343] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: 727cacd242844455915bd9963d2db19f
I20260505 07:54:20.120536 718 leader_election.cc:304] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:20.120952 2447 tablet_replica.cc:333] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.121199 2447 raft_consensus.cc:2243] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.121330 3199 raft_consensus.cc:2749] T 727cacd242844455915bd9963d2db19f P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260505 07:54:20.121393 2447 raft_consensus.cc:2272] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.121594 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59a3cd77808c4e858ea2ef9ed2a28c38" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.121779 912 raft_consensus.cc:2376] T 59a3cd77808c4e858ea2ef9ed2a28c38 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1. Re-sending same reply.
I20260505 07:54:20.121978 2447 ts_tablet_manager.cc:1916] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.125290 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "172ca66c49354f549fa55fe63abf9970" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.125798 717 leader_election.cc:304] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:20.126111 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 3075e87e078f4d2aa1d329d18902354b (table default.Table_c1cf7717be07400aacdab563b4255450 [id=4d79fded98df46cc884cabdc3fba5d22]) successfully deleted
I20260505 07:54:20.126199 3364 raft_consensus.cc:3060] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.126751 2276 ts_tablet_manager.cc:1929] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260505 07:54:20.126879 2276 log.cc:1199] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/02716f62aa354d8f839266661b7f05ea
I20260505 07:54:20.127345 2276 ts_tablet_manager.cc:1950] T 02716f62aa354d8f839266661b7f05ea P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.128234 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 218) took 9074 ms. Trace:
I20260505 07:54:20.128336 2276 rpcz_store.cc:276] 0505 07:54:11.053952 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.054001 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:20.073981 (+9019980us) ts_tablet_manager.cc:1151] Deleting tablet 02716f62aa354d8f839266661b7f05ea
0505 07:54:20.073994 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.094240 (+ 20246us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.126669 (+ 32429us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.128193 (+ 1524us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9019759}
I20260505 07:54:20.128927 3364 raft_consensus.cc:2749] T 172ca66c49354f549fa55fe63abf9970 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.136559 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 221) took 9013 ms. Trace:
I20260505 07:54:20.136657 2276 rpcz_store.cc:276] 0505 07:54:11.123397 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.123449 (+ 52us) service_pool.cc:224] Handling call
0505 07:54:20.129014 (+9005565us) ts_tablet_manager.cc:1151] Deleting tablet bde59d52fd234343b1ca13a7d4aae6bf
0505 07:54:20.135771 (+ 6757us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.136516 (+ 745us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9005340}
W20260505 07:54:20.136915 718 consensus_peers.cc:597] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9b7c6e9463342a5a73af12da4447f89. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:20.137472 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet bde59d52fd234343b1ca13a7d4aae6bf because the tablet was not found. No further retry: Not found: Tablet not found: bde59d52fd234343b1ca13a7d4aae6bf
I20260505 07:54:20.138515 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 220) took 9016 ms. Trace:
I20260505 07:54:20.138612 2276 rpcz_store.cc:276] 0505 07:54:11.122156 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.123517 (+ 1361us) service_pool.cc:224] Handling call
0505 07:54:20.138398 (+9014881us) ts_tablet_manager.cc:1151] Deleting tablet a32eeaca77c1475f879465cb1046543b
0505 07:54:20.138407 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.138483 (+ 76us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9014709}
W20260505 07:54:20.139025 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a32eeaca77c1475f879465cb1046543b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a32eeaca77c1475f879465cb1046543b already in progress: creating tablet
I20260505 07:54:20.139385 2447 ts_tablet_manager.cc:1929] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.139505 2447 log.cc:1199] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/27bacc97d7534284970f218cf68ac75b
I20260505 07:54:20.139951 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_42d8c6c76f0444e6ac4fd58932a8b92e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:20.140511 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_42d8c6c76f0444e6ac4fd58932a8b92e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260505 07:54:20.143535 717 consensus_peers.cc:597] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6546745e894545ff9c5f91bcc6e1a41e. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:20.144285 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 222) took 9019 ms. Trace:
I20260505 07:54:20.144390 2276 rpcz_store.cc:276] 0505 07:54:11.124659 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.124720 (+ 61us) service_pool.cc:224] Handling call
0505 07:54:20.144192 (+9019472us) ts_tablet_manager.cc:1151] Deleting tablet 17327cee69a24e0cb493ff55f948bf6c
0505 07:54:20.144201 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.144252 (+ 51us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9019285}
W20260505 07:54:20.144788 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 17327cee69a24e0cb493ff55f948bf6c because the tablet was not found. No further retry: Not found: Tablet not found: 17327cee69a24e0cb493ff55f948bf6c
I20260505 07:54:20.149799 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 233) took 8123 ms. Trace:
I20260505 07:54:20.151041 1181 tablet_bootstrap.cc:492] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.151250 1181 ts_tablet_manager.cc:1403] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.049s user 0.005s sys 0.000s
I20260505 07:54:20.152199 1181 raft_consensus.cc:359] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.152444 1181 raft_consensus.cc:385] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.152464 3261 raft_consensus.cc:493] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.152633 3261 raft_consensus.cc:515] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.153285 3261 leader_election.cc:290] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:20.154301 1181 raft_consensus.cc:740] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.154578 1181 consensus_queue.cc:260] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.154773 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efb07cb60f64dfead739a3ca1343df9" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.154822 3261 consensus_queue.cc:1048] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.155131 1181 ts_tablet_manager.cc:1434] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260505 07:54:20.155656 1181 tablet_bootstrap.cc:492] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.156373 3364 consensus_queue.cc:1048] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.154067 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6efb07cb60f64dfead739a3ca1343df9" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.157727 912 raft_consensus.cc:2393] T 6efb07cb60f64dfead739a3ca1343df9 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate eac7e1e8caba43cdb4df80cbb80e891c in this term.
I20260505 07:54:20.156647 3144 raft_consensus.cc:493] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.158031 3144 raft_consensus.cc:515] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.158315 991 leader_election.cc:304] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.158627 3261 raft_consensus.cc:3060] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.160979 3261 raft_consensus.cc:2749] T 6efb07cb60f64dfead739a3ca1343df9 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.162874 1352 catalog_manager.cc:6216] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: default.Table_79956c61f6444a4fad1ca4b7950feb9d [id=b131fb4265684cd2b6506a7abeb29871] alter complete (version 2)
I20260505 07:54:20.167352 3203 raft_consensus.cc:493] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.167598 3203 raft_consensus.cc:515] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.168515 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b48208e68247e78c12f0fec7ba07dc" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.168434 2276 rpcz_store.cc:276] 0505 07:54:12.026206 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.026254 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:20.149663 (+8123409us) ts_tablet_manager.cc:1151] Deleting tablet f57e14ee70584d258c83ff1b01189713
0505 07:54:20.149675 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.149761 (+ 86us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8123247}
W20260505 07:54:20.168995 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet f57e14ee70584d258c83ff1b01189713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f57e14ee70584d258c83ff1b01189713 already in progress: creating tablet
I20260505 07:54:20.169060 3203 leader_election.cc:290] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:20.169468 988 leader_election.cc:343] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.139873 2447 ts_tablet_manager.cc:1950] T 27bacc97d7534284970f218cf68ac75b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.169742 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b48208e68247e78c12f0fec7ba07dc" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.169991 912 raft_consensus.cc:2468] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:20.170083 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 222) took 10362 ms. Trace:
I20260505 07:54:20.170382 991 leader_election.cc:304] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.170734 3203 raft_consensus.cc:2804] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:20.170866 3203 raft_consensus.cc:493] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:20.170987 3203 raft_consensus.cc:3060] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.173867 3261 consensus_queue.cc:1048] T 1506cd31fe364e328867bbf70e36845e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.174435 2276 tablet_replica.cc:333] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.174393 3203 raft_consensus.cc:515] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.174657 2276 raft_consensus.cc:2243] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.174820 2276 raft_consensus.cc:2272] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.175407 2276 ts_tablet_manager.cc:1916] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:20.184036 444 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.60:35401, user_credentials={real_user=slave}} blocked reactor thread for 105336us
I20260505 07:54:20.187547 1181 tablet_bootstrap.cc:654] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.188287 3261 consensus_queue.cc:1048] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.190478 3203 leader_election.cc:290] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:20.170188 2447 rpcz_store.cc:276] 0505 07:54:09.808000 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.808040 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:20.120868 (+10312828us) ts_tablet_manager.cc:1151] Deleting tablet 27bacc97d7534284970f218cf68ac75b
0505 07:54:20.120879 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.134068 (+ 13189us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.139318 (+ 5250us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.170047 (+ 30729us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10312734}
I20260505 07:54:20.191313 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 224) took 10383 ms. Trace:
I20260505 07:54:20.191484 2447 rpcz_store.cc:276] 0505 07:54:09.808201 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.809066 (+ 865us) service_pool.cc:224] Handling call
0505 07:54:20.191084 (+10382018us) ts_tablet_manager.cc:1151] Deleting tablet 05351965bdba4ccca8d13223cd505948
0505 07:54:20.191094 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.191180 (+ 86us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10381893}
I20260505 07:54:20.191737 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b48208e68247e78c12f0fec7ba07dc" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:20.192029 2447 tablet_replica.cc:333] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.192270 2447 raft_consensus.cc:2243] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.192673 2447 raft_consensus.cc:2272] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.192727 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b48208e68247e78c12f0fec7ba07dc" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:20.192698 912 raft_consensus.cc:3060] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.193255 2447 ts_tablet_manager.cc:1916] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:20.193220 444 leader_election.cc:336] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401): Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111)
I20260505 07:54:20.193548 444 leader_election.cc:304] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 379cdfdecc3341ea9d7d9dc09274338d; no voters: 23c57a2c927c4c54bccecfeb3a775311, 28d3eda04f9c47ae976d19c218b6dcc5
I20260505 07:54:20.194000 3378 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 379cdfdecc3341ea9d7d9dc09274338d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260505 07:54:20.195309 912 raft_consensus.cc:2468] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
W20260505 07:54:20.195997 988 leader_election.cc:343] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.201190 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 27bacc97d7534284970f218cf68ac75b (table default.Table_db40b13ba59549c6bae350382395bceb [id=db770342dce84ba1ab28d029ddb65d78]) successfully deleted
W20260505 07:54:20.202464 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 05351965bdba4ccca8d13223cd505948 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 05351965bdba4ccca8d13223cd505948 already in progress: creating tablet
I20260505 07:54:20.200446 3361 consensus_queue.cc:1048] T 0e08cf03ea0b495a94c6566ea6c8caad P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.210542 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 321) took 5523 ms. Trace:
I20260505 07:54:20.210739 2268 rpcz_store.cc:276] 0505 07:54:14.686630 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.686670 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:20.210261 (+5523591us) ts_tablet_manager.cc:1151] Deleting tablet e29ba55a846548679cb3b86305d0d6ed
0505 07:54:20.210276 (+ 15us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.210480 (+ 204us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5523381}
I20260505 07:54:20.211575 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 324) took 5524 ms. Trace:
I20260505 07:54:20.212647 1181 tablet_bootstrap.cc:492] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.212814 1181 ts_tablet_manager.cc:1403] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.057s user 0.003s sys 0.000s
I20260505 07:54:20.213891 1181 raft_consensus.cc:359] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.214116 1181 raft_consensus.cc:385] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.214179 1181 raft_consensus.cc:740] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.214444 1181 consensus_queue.cc:260] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.216147 1181 ts_tablet_manager.cc:1434] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:20.216491 1181 tablet_bootstrap.cc:492] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.218637 991 leader_election.cc:304] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.219188 1181 tablet_bootstrap.cc:654] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:20.219221 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet e29ba55a846548679cb3b86305d0d6ed because the tablet was not found. No further retry: Not found: Tablet not found: e29ba55a846548679cb3b86305d0d6ed
I20260505 07:54:20.219198 3203 raft_consensus.cc:2804] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:20.219686 3203 raft_consensus.cc:697] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:20.219951 3203 consensus_queue.cc:237] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.227842 2276 ts_tablet_manager.cc:1929] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.227993 2276 log.cc:1199] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/b0d966bcd7f1419cbd090afc185ae78a
I20260505 07:54:20.228344 2276 ts_tablet_manager.cc:1950] T b0d966bcd7f1419cbd090afc185ae78a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.229234 2447 ts_tablet_manager.cc:1929] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.229233 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 234) took 8191 ms. Trace:
I20260505 07:54:20.229327 2276 rpcz_store.cc:276] 0505 07:54:12.037922 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.037972 (+ 50us) service_pool.cc:224] Handling call
0505 07:54:20.174378 (+8136406us) ts_tablet_manager.cc:1151] Deleting tablet b0d966bcd7f1419cbd090afc185ae78a
0505 07:54:20.174391 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.178904 (+ 4513us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.226527 (+ 47623us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.229196 (+ 2669us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8135590}
I20260505 07:54:20.212512 2268 rpcz_store.cc:276] 0505 07:54:14.687361 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.687402 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:20.211384 (+5523982us) ts_tablet_manager.cc:1151] Deleting tablet 36b33d265ba04e93993c7e0a4efea1a4
0505 07:54:20.211394 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.211521 (+ 127us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5523780}
I20260505 07:54:20.230469 2447 log.cc:1199] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/6fe19306f7df49fdb771d21f6560e697
I20260505 07:54:20.230505 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 325) took 5542 ms. Trace:
I20260505 07:54:20.230603 2268 rpcz_store.cc:276] 0505 07:54:14.687698 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.687740 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:20.230408 (+5542668us) ts_tablet_manager.cc:1151] Deleting tablet d3d11bd19f074bbab101448b45426b7a
0505 07:54:20.230420 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.230489 (+ 69us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5542495}
I20260505 07:54:20.231079 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 326) took 5542 ms. Trace:
I20260505 07:54:20.231173 2268 rpcz_store.cc:276] 0505 07:54:14.688222 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.688260 (+ 38us) service_pool.cc:224] Handling call
0505 07:54:20.231005 (+5542745us) ts_tablet_manager.cc:1151] Deleting tablet 4258df74ea0841449846cdf1f6e00b9b
0505 07:54:20.231016 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.231067 (+ 51us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5542574}
I20260505 07:54:20.231629 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 322) took 5544 ms. Trace:
I20260505 07:54:20.231729 2268 rpcz_store.cc:276] 0505 07:54:14.687004 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.687042 (+ 38us) service_pool.cc:224] Handling call
0505 07:54:20.231529 (+5544487us) ts_tablet_manager.cc:1151] Deleting tablet 46274a44a06b4306b5dac43c66c85a80
0505 07:54:20.231540 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.231615 (+ 75us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5542975}
I20260505 07:54:20.232064 2447 ts_tablet_manager.cc:1950] T 6fe19306f7df49fdb771d21f6560e697 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.232138 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 328) took 5538 ms. Trace:
I20260505 07:54:20.232229 2268 rpcz_store.cc:276] 0505 07:54:14.694129 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.695020 (+ 891us) service_pool.cc:224] Handling call
0505 07:54:20.232063 (+5537043us) ts_tablet_manager.cc:1151] Deleting tablet 43e4c92b7c3140fdb2d6a290aaf0ad55
0505 07:54:20.232072 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.232122 (+ 50us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5536868}
I20260505 07:54:20.232646 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 331) took 5536 ms. Trace:
I20260505 07:54:20.232808 2268 rpcz_store.cc:276] 0505 07:54:14.696413 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.696582 (+ 169us) service_pool.cc:224] Handling call
0505 07:54:20.232569 (+5535987us) ts_tablet_manager.cc:1151] Deleting tablet 3d6494590052459485059f5dce46f34f
0505 07:54:20.232580 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.232632 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5535836}
I20260505 07:54:20.233225 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 332) took 5530 ms. Trace:
I20260505 07:54:20.233307 2268 rpcz_store.cc:276] 0505 07:54:14.702434 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.702474 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:20.233147 (+5530673us) ts_tablet_manager.cc:1151] Deleting tablet 32c008bd99224a7595cecc607e5496a9
0505 07:54:20.233157 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.233211 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5530538}
I20260505 07:54:20.233675 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 333) took 5530 ms. Trace:
I20260505 07:54:20.232725 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 230) took 10321 ms. Trace:
I20260505 07:54:20.233764 2268 rpcz_store.cc:276] 0505 07:54:14.703217 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.703256 (+ 39us) service_pool.cc:224] Handling call
0505 07:54:20.233609 (+5530353us) ts_tablet_manager.cc:1151] Deleting tablet a7fe9c7199cc4039ae877dc3d4aab17e
0505 07:54:20.233617 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.233662 (+ 45us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5530217}
I20260505 07:54:20.233868 2447 rpcz_store.cc:276] 0505 07:54:09.911453 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:09.912327 (+ 874us) service_pool.cc:224] Handling call
0505 07:54:20.191947 (+10279620us) ts_tablet_manager.cc:1151] Deleting tablet 6fe19306f7df49fdb771d21f6560e697
0505 07:54:20.191958 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.207605 (+ 15647us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.229151 (+ 21546us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.232681 (+ 3530us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10279410}
W20260505 07:54:20.234407 718 consensus_peers.cc:597] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f842d8ac6b2142ffa69945c6087be34b. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:20.239091 2447 tablet_replica.cc:333] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.239539 2447 raft_consensus.cc:2243] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.240173 2447 raft_consensus.cc:2272] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.242390 2447 ts_tablet_manager.cc:1916] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.247234 3173 consensus_queue.cc:1048] T e243711164f04faabbde53b0a09d90b1 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.252238 3366 consensus_queue.cc:1048] T 7235a3ccef904913bbd67c403aac3a87 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.252368 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05351965bdba4ccca8d13223cd505948" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:20.252895 856 leader_election.cc:343] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.253856 3361 consensus_queue.cc:1048] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.254835 3144 leader_election.cc:290] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:20.255216 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 334) took 5550 ms. Trace:
I20260505 07:54:20.255312 2268 rpcz_store.cc:276] 0505 07:54:14.704328 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.704374 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:20.255137 (+5550763us) ts_tablet_manager.cc:1151] Deleting tablet 4783dd0b2fbe40219852d850fc0d9a82
0505 07:54:20.255146 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.255201 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5550535}
I20260505 07:54:20.255793 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 337) took 5547 ms. Trace:
I20260505 07:54:20.255898 2268 rpcz_store.cc:276] 0505 07:54:14.708373 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.708415 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:20.255682 (+5547267us) ts_tablet_manager.cc:1151] Deleting tablet a0f2f46d0e7840458cb992ce2a2dbea2
0505 07:54:20.255695 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.255774 (+ 79us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5547102}
I20260505 07:54:20.256372 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 338) took 5526 ms. Trace:
I20260505 07:54:20.256469 2268 rpcz_store.cc:276] 0505 07:54:14.729999 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.730050 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:20.256285 (+5526235us) ts_tablet_manager.cc:1151] Deleting tablet 1838f89dc14a47f99f95f4cc5fdefd70
0505 07:54:20.256296 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.256356 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5525934}
I20260505 07:54:20.256932 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 339) took 5523 ms. Trace:
I20260505 07:54:20.257023 2268 rpcz_store.cc:276] 0505 07:54:14.733579 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.733623 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:20.256849 (+5523226us) ts_tablet_manager.cc:1151] Deleting tablet 7585f00119e14006990ed551667b4486
0505 07:54:20.256859 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.256917 (+ 58us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5523017}
I20260505 07:54:20.257452 2268 tablet_replica.cc:333] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:20.257688 2268 raft_consensus.cc:2243] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.258065 2268 raft_consensus.cc:2272] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:20.258303 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 36b33d265ba04e93993c7e0a4efea1a4 because the tablet was not found. No further retry: Not found: Tablet not found: 36b33d265ba04e93993c7e0a4efea1a4
I20260505 07:54:20.258407 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05351965bdba4ccca8d13223cd505948" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:20.258630 1318 catalog_manager.cc:4784] Async tablet task AlterTable RPC for tablet 24f4854e8752493fb832ae2c12a033ce (table default.Table_018af517d4374faaa03e4eba33184d90 [id=1a1c2d94983e409c8b75400de8adbae8], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 24f4854e8752493fb832ae2c12a033ce consensus state has no leader
W20260505 07:54:20.258953 854 leader_election.cc:343] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.259104 1176 tablet_bootstrap.cc:492] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:20.259107 854 leader_election.cc:304] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.259258 1176 ts_tablet_manager.cc:1403] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.188s user 0.016s sys 0.027s
I20260505 07:54:20.259315 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 6fe19306f7df49fdb771d21f6560e697 (table default.Table_9e1b44e1d482455e8cc700e5de6c4a51 [id=7963a4fd0d0e49bb96e46969d8ce5e93]) successfully deleted
I20260505 07:54:20.259477 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet b0d966bcd7f1419cbd090afc185ae78a (table default.Table_58eed87aa11e4e3fa7a7d18a135128a0 [id=3ae3bc457dcb4db1b5f4360cb9e5c81c]) successfully deleted
W20260505 07:54:20.259896 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet d3d11bd19f074bbab101448b45426b7a because the tablet was not found. No further retry: Not found: Tablet not found: d3d11bd19f074bbab101448b45426b7a
I20260505 07:54:20.260121 1176 raft_consensus.cc:359] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.260279 1176 raft_consensus.cc:385] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260505 07:54:20.260344 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 4258df74ea0841449846cdf1f6e00b9b because the tablet was not found. No further retry: Not found: Tablet not found: 4258df74ea0841449846cdf1f6e00b9b
I20260505 07:54:20.260353 1176 raft_consensus.cc:740] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.260586 1176 consensus_queue.cc:260] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:20.260880 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 46274a44a06b4306b5dac43c66c85a80 because the tablet was not found. No further retry: Not found: Tablet not found: 46274a44a06b4306b5dac43c66c85a80
W20260505 07:54:20.261089 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 43e4c92b7c3140fdb2d6a290aaf0ad55 because the tablet was not found. No further retry: Not found: Tablet not found: 43e4c92b7c3140fdb2d6a290aaf0ad55
W20260505 07:54:20.261310 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 3d6494590052459485059f5dce46f34f because the tablet was not found. No further retry: Not found: Tablet not found: 3d6494590052459485059f5dce46f34f
W20260505 07:54:20.261493 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 32c008bd99224a7595cecc607e5496a9 because the tablet was not found. No further retry: Not found: Tablet not found: 32c008bd99224a7595cecc607e5496a9
W20260505 07:54:20.261698 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet a7fe9c7199cc4039ae877dc3d4aab17e because the tablet was not found. No further retry: Not found: Tablet not found: a7fe9c7199cc4039ae877dc3d4aab17e
W20260505 07:54:20.261896 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 4783dd0b2fbe40219852d850fc0d9a82 because the tablet was not found. No further retry: Not found: Tablet not found: 4783dd0b2fbe40219852d850fc0d9a82
W20260505 07:54:20.262110 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet a0f2f46d0e7840458cb992ce2a2dbea2 because the tablet was not found. No further retry: Not found: Tablet not found: a0f2f46d0e7840458cb992ce2a2dbea2
W20260505 07:54:20.262310 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 1838f89dc14a47f99f95f4cc5fdefd70 because the tablet was not found. No further retry: Not found: Tablet not found: 1838f89dc14a47f99f95f4cc5fdefd70
W20260505 07:54:20.262516 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 7585f00119e14006990ed551667b4486 because the tablet was not found. No further retry: Not found: Tablet not found: 7585f00119e14006990ed551667b4486
I20260505 07:54:20.262708 1176 ts_tablet_manager.cc:1434] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:20.262885 2276 tablet_replica.cc:333] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.263074 2276 raft_consensus.cc:2243] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.263103 1176 tablet_bootstrap.cc:492] T 6ea1d3829ede4b5582a022b64265df7e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:20.263216 2276 raft_consensus.cc:2272] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.263728 2276 ts_tablet_manager.cc:1916] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.267390 1176 tablet_bootstrap.cc:654] T 6ea1d3829ede4b5582a022b64265df7e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.267871 3144 raft_consensus.cc:2749] T 05351965bdba4ccca8d13223cd505948 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:20.271179 718 consensus_peers.cc:597] T 6546745e894545ff9c5f91bcc6e1a41e P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6546745e894545ff9c5f91bcc6e1a41e. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:20.271225 2447 ts_tablet_manager.cc:1929] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:20.271358 2447 log.cc:1199] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/5683d7366de84653ace56f0edcd3818c
I20260505 07:54:20.271817 2447 ts_tablet_manager.cc:1950] T 5683d7366de84653ace56f0edcd3818c P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.272673 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 231) took 9920 ms. Trace:
I20260505 07:54:20.272775 2447 rpcz_store.cc:276] 0505 07:54:10.352050 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.352103 (+ 53us) service_pool.cc:224] Handling call
0505 07:54:20.239009 (+9886906us) ts_tablet_manager.cc:1151] Deleting tablet 5683d7366de84653ace56f0edcd3818c
0505 07:54:20.239028 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.245961 (+ 6933us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.271129 (+ 25168us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.272631 (+ 1502us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9886715}
I20260505 07:54:20.274570 2268 ts_tablet_manager.cc:1916] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.276320 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 5683d7366de84653ace56f0edcd3818c (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]) successfully deleted
I20260505 07:54:20.276396 1181 tablet_bootstrap.cc:492] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.276542 1181 ts_tablet_manager.cc:1403] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.060s user 0.003s sys 0.000s
I20260505 07:54:20.277146 2447 tablet_replica.cc:333] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.277442 2447 raft_consensus.cc:2243] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.277405 1181 raft_consensus.cc:359] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.277683 1181 raft_consensus.cc:385] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.277791 2447 raft_consensus.cc:2272] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.277809 1181 raft_consensus.cc:740] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.278182 1181 consensus_queue.cc:260] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.278326 2447 ts_tablet_manager.cc:1916] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.278924 1181 ts_tablet_manager.cc:1434] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:20.279275 1181 tablet_bootstrap.cc:492] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.281780 1181 tablet_bootstrap.cc:654] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.291441 1176 tablet_bootstrap.cc:492] T 6ea1d3829ede4b5582a022b64265df7e P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:20.291640 1176 ts_tablet_manager.cc:1403] T 6ea1d3829ede4b5582a022b64265df7e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.029s user 0.004s sys 0.002s
I20260505 07:54:20.294442 1176 raft_consensus.cc:359] T 6ea1d3829ede4b5582a022b64265df7e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.294677 1176 raft_consensus.cc:385] T 6ea1d3829ede4b5582a022b64265df7e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.294770 1176 raft_consensus.cc:740] T 6ea1d3829ede4b5582a022b64265df7e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.295001 1176 consensus_queue.cc:260] T 6ea1d3829ede4b5582a022b64265df7e P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.295838 1176 ts_tablet_manager.cc:1434] T 6ea1d3829ede4b5582a022b64265df7e P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260505 07:54:20.296123 1176 tablet_bootstrap.cc:492] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:20.295125 3202 consensus_queue.cc:1048] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.298736 1176 tablet_bootstrap.cc:654] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.302193 2268 ts_tablet_manager.cc:1929] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.302299 2268 log.cc:1199] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/70051011024f4971bd0166d335d32393
I20260505 07:54:20.303462 2268 ts_tablet_manager.cc:1950] T 70051011024f4971bd0166d335d32393 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:20.304239 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 354) took 5127 ms. Trace:
I20260505 07:54:20.304325 2268 rpcz_store.cc:276] 0505 07:54:15.176664 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:15.176712 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:20.257384 (+5080672us) ts_tablet_manager.cc:1151] Deleting tablet 70051011024f4971bd0166d335d32393
0505 07:54:20.257396 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.281728 (+ 24332us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.293780 (+ 12052us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.304222 (+ 10442us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5080485}
I20260505 07:54:20.304852 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 70051011024f4971bd0166d335d32393 (table default.Table_6b9d99bdf8624bba90ba61af22f36dc2 [id=6fa686898f454d2aa691814ad1284d0f]) successfully deleted
I20260505 07:54:20.305034 2447 ts_tablet_manager.cc:1929] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.305138 2447 log.cc:1199] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/df3a49b59ed6485f970074391f932482
I20260505 07:54:20.305511 2447 ts_tablet_manager.cc:1950] T df3a49b59ed6485f970074391f932482 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.306246 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 244) took 9643 ms. Trace:
I20260505 07:54:20.306344 2447 rpcz_store.cc:276] 0505 07:54:10.662843 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.663014 (+ 171us) service_pool.cc:224] Handling call
0505 07:54:20.275413 (+9612399us) ts_tablet_manager.cc:1151] Deleting tablet df3a49b59ed6485f970074391f932482
0505 07:54:20.275426 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.288848 (+ 13422us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.304958 (+ 16110us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.306183 (+ 1225us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9612244}
I20260505 07:54:20.306699 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet df3a49b59ed6485f970074391f932482 (table default.Table_7af623f581af498b86c8c3b1adbc6b80 [id=306149ae47a745edabf863d5004d5012]) successfully deleted
I20260505 07:54:20.306756 2268 tablet_replica.cc:333] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:20.306937 2268 raft_consensus.cc:2243] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.307226 2268 raft_consensus.cc:2272] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.314445 3361 consensus_queue.cc:1048] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.318202 1172 tablet_bootstrap.cc:492] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.318333 1172 ts_tablet_manager.cc:1403] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.289s user 0.030s sys 0.034s
I20260505 07:54:20.318962 1172 raft_consensus.cc:359] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.319113 1172 raft_consensus.cc:385] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.319167 1172 raft_consensus.cc:740] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.320935 1172 consensus_queue.cc:260] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.322780 1172 ts_tablet_manager.cc:1434] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260505 07:54:20.324040 1172 tablet_bootstrap.cc:492] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:20.324141 2447 tablet_replica.cc:333] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.324368 2447 raft_consensus.cc:2243] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.324596 2447 raft_consensus.cc:2272] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.324944 2268 ts_tablet_manager.cc:1916] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.325506 2276 ts_tablet_manager.cc:1929] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.325649 2276 log.cc:1199] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/b8a85b65b1904bd6b793af48f71ed726
I20260505 07:54:20.325822 1172 tablet_bootstrap.cc:654] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.326120 2276 ts_tablet_manager.cc:1950] T b8a85b65b1904bd6b793af48f71ed726 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.326443 3203 raft_consensus.cc:493] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.326591 3203 raft_consensus.cc:515] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.327136 3203 leader_election.cc:290] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:20.327314 2447 ts_tablet_manager.cc:1916] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.327742 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7071df211bd34b86a751142bae3271b2" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.327971 782 raft_consensus.cc:2393] T 7071df211bd34b86a751142bae3271b2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d63a3e535f8e4dd4a88e447253405b77 in current term 1: Already voted for candidate d2799f5c6b4f4336b976ca9bbb458533 in this term.
I20260505 07:54:20.328073 1181 tablet_bootstrap.cc:492] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.328181 1181 ts_tablet_manager.cc:1403] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.049s user 0.004s sys 0.000s
I20260505 07:54:20.328177 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7071df211bd34b86a751142bae3271b2" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.328616 991 leader_election.cc:304] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.328857 1181 raft_consensus.cc:359] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.328989 1181 raft_consensus.cc:385] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.329036 1181 raft_consensus.cc:740] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.329248 3173 raft_consensus.cc:3060] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.329206 1181 consensus_queue.cc:260] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.329608 1181 ts_tablet_manager.cc:1434] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:20.329836 1181 tablet_bootstrap.cc:492] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.331272 3254 consensus_queue.cc:1048] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.331634 1181 tablet_bootstrap.cc:654] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.331741 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 235) took 8283 ms. Trace:
I20260505 07:54:20.331832 2276 rpcz_store.cc:276] 0505 07:54:12.048562 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.049197 (+ 635us) service_pool.cc:224] Handling call
0505 07:54:20.262835 (+8213638us) ts_tablet_manager.cc:1151] Deleting tablet b8a85b65b1904bd6b793af48f71ed726
0505 07:54:20.262847 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.266679 (+ 3832us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.325377 (+ 58698us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.331723 (+ 6346us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":8213480}
I20260505 07:54:20.332175 2276 tablet_replica.cc:333] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.332363 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet b8a85b65b1904bd6b793af48f71ed726 (table default.Table_a8605cf7bc3b44f39a895bfb7ad9934b [id=daf1e66deafd417787948bb18d615391]) successfully deleted
I20260505 07:54:20.332402 2276 raft_consensus.cc:2243] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.332506 2276 pending_rounds.cc:70] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: Trying to abort 1 pending ops.
I20260505 07:54:20.332551 2276 pending_rounds.cc:77] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555533045260288 op_type: NO_OP noop_request { }
I20260505 07:54:20.332625 2276 raft_consensus.cc:2889] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:20.332695 2276 raft_consensus.cc:2272] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:20.332731 991 consensus_peers.cc:597] T f840d5f58161468385d5c4f116b78c3d P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:20.333554 1172 tablet_bootstrap.cc:492] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.333653 1172 ts_tablet_manager.cc:1403] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.001s
I20260505 07:54:20.333690 3173 raft_consensus.cc:2749] T 7071df211bd34b86a751142bae3271b2 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.334219 2276 ts_tablet_manager.cc:1916] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.334223 1172 raft_consensus.cc:359] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.334389 1172 raft_consensus.cc:385] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.334437 1172 raft_consensus.cc:740] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.334625 1172 consensus_queue.cc:260] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.335712 1172 ts_tablet_manager.cc:1434] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:20.336459 1172 tablet_bootstrap.cc:492] T 3fece4034c3a42169fae0c9f41dce0bb P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:20.337796 1172 tablet_bootstrap.cc:654] T 3fece4034c3a42169fae0c9f41dce0bb P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.342586 2276 ts_tablet_manager.cc:1929] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.342718 2276 log.cc:1199] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/f840d5f58161468385d5c4f116b78c3d
I20260505 07:54:20.343154 2276 ts_tablet_manager.cc:1950] T f840d5f58161468385d5c4f116b78c3d P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.343501 1176 tablet_bootstrap.cc:492] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:20.343708 1176 ts_tablet_manager.cc:1403] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.048s user 0.004s sys 0.000s
I20260505 07:54:20.344661 1176 raft_consensus.cc:359] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.344919 1176 raft_consensus.cc:385] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.345079 1176 raft_consensus.cc:740] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.345396 1176 consensus_queue.cc:260] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.346238 1176 ts_tablet_manager.cc:1434] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:20.346732 1176 tablet_bootstrap.cc:492] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:20.348773 1176 tablet_bootstrap.cc:654] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.352222 1354 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 75de7522fa534140822ead8be4dc2b2d
I20260505 07:54:20.353214 1354 catalog_manager.cc:6741] ReplaceTablet: tablet 75de7522fa534140822ead8be4dc2b2d deleted and replaced by tablet 8e1e4541c93342aca228008a6894522e
I20260505 07:54:20.353358 1354 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 273) took 1737 ms. Trace:
I20260505 07:54:20.353682 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 75de7522fa534140822ead8be4dc2b2d with delete_type TABLET_DATA_DELETED (replaced by tablet 8e1e4541c93342aca228008a6894522e) from {username='slave'} at 127.0.0.1:53998
I20260505 07:54:20.353447 1354 rpcz_store.cc:276] 0505 07:54:18.615688 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:19.888195 (+1272507us) service_pool.cc:224] Handling call
0505 07:54:19.891803 (+ 3608us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639e385b 00007fd65a477107 00007fd65a476e35 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:20.353304 (+461501us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:20.142604 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:20.142698 (+ 94us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:20.142703 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:20.142705 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:20.142816 (+ 111us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:20.142827 (+ 11us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:20.142831 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:20.142914 (+ 83us) tablet.cc:701] Row locks acquired
0505 07:54:20.142917 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:20.142950 (+ 33us) write_op.cc:270] Start()
0505 07:54:20.142982 (+ 32us) write_op.cc:276] Timestamp: P: 1777967660142911 usec, L: 0
0505 07:54:20.142985 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:20.143129 (+ 144us) log.cc:844] Serialized 622 byte log entry
0505 07:54:20.169281 (+ 26152us) op_driver.cc:464] REPLICATION: finished
0505 07:54:20.169565 (+ 284us) write_op.cc:301] APPLY: starting
0505 07:54:20.169580 (+ 15us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:20.169621 (+ 41us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:20.169667 (+ 46us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:20.169789 (+ 122us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:20.169853 (+ 64us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:20.169865 (+ 12us) write_op.cc:312] APPLY: finished
0505 07:54:20.169934 (+ 69us) log.cc:844] Serialized 29 byte log entry
0505 07:54:20.170154 (+ 220us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.170171 (+ 17us) write_op.cc:454] Released schema lock
0505 07:54:20.170179 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:20.170359 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.170363 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":7540096,"child_traces":[["op",{"apply.queue_time_us":232,"apply.run_cpu_time_us":528,"apply.run_wall_time_us":757,"num_ops":2,"prepare.queue_time_us":11905,"prepare.run_cpu_time_us":629,"prepare.run_wall_time_us":1351,"raft.queue_time_us":35,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":26260}]]}
I20260505 07:54:20.354831 1355 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 276) took 1731 ms. Trace:
I20260505 07:54:20.354923 1355 rpcz_store.cc:276] 0505 07:54:18.623107 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:20.101592 (+1478485us) service_pool.cc:224] Handling call
0505 07:54:20.139878 (+ 38286us) spinlock_profiling.cc:243] Waited 38 ms on lock 0x616000041854. stack: 00007fd64fbd0086 00007fd64fbcfe40 00007fd64eb893ad 000000000099adb4 000000000099ad7d 00007fd65ef586bd 00007fd65ef587b3 00007fd65bb16b21 00007fd66384abad 00007fd6639d6b13 00007fd65a46a7c6 00007fd65a46a4f5 00007fd656bc310e 00007fd656bc1fee 00007fd656bc8228 00007fd656bc8b8f
0505 07:54:20.140599 (+ 721us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:20.141000 (+ 401us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:20.290915 (+149915us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:20.354637 (+ 63722us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:20.354746 (+ 109us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:20.145865 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:20.145947 (+ 82us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:20.145952 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:20.145955 (+ 3us) tablet.cc:662] Decoding operations
0505 07:54:20.146024 (+ 69us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:20.146031 (+ 7us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:20.146035 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:20.146094 (+ 59us) tablet.cc:701] Row locks acquired
0505 07:54:20.146096 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:20.146119 (+ 23us) write_op.cc:270] Start()
0505 07:54:20.146141 (+ 22us) write_op.cc:276] Timestamp: P: 1777967660146083 usec, L: 0
0505 07:54:20.146146 (+ 5us) op_driver.cc:348] REPLICATION: starting
0505 07:54:20.146261 (+ 115us) log.cc:844] Serialized 433 byte log entry
0505 07:54:20.288090 (+141829us) op_driver.cc:464] REPLICATION: finished
0505 07:54:20.288450 (+ 360us) write_op.cc:301] APPLY: starting
0505 07:54:20.288466 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:20.289590 (+ 1124us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:20.289655 (+ 65us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:20.289858 (+ 203us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:20.289894 (+ 36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:20.289902 (+ 8us) write_op.cc:312] APPLY: finished
0505 07:54:20.290542 (+ 640us) log.cc:844] Serialized 29 byte log entry
0505 07:54:20.290799 (+ 257us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.290824 (+ 25us) write_op.cc:454] Released schema lock
0505 07:54:20.290832 (+ 8us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:20.291084 (+ 252us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.291088 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":87345024,"child_traces":[["op",{"apply.queue_time_us":249,"apply.run_cpu_time_us":766,"apply.run_wall_time_us":2614,"num_ops":2,"prepare.queue_time_us":4469,"prepare.run_cpu_time_us":473,"prepare.run_wall_time_us":729,"raft.queue_time_us":29,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":141910}]]}
I20260505 07:54:20.355713 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_353ee2d7958f4f64b9e2697f7161f11e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:20.356240 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_353ee2d7958f4f64b9e2697f7161f11e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:20.358758 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 75de7522fa534140822ead8be4dc2b2d with delete_type TABLET_DATA_DELETED (replaced by tablet 8e1e4541c93342aca228008a6894522e) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:20.365044 1355 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 277) took 1740 ms. Trace:
I20260505 07:54:20.365145 1355 rpcz_store.cc:276] 0505 07:54:18.624700 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:20.355502 (+1730802us) service_pool.cc:224] Handling call
0505 07:54:20.356335 (+ 833us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:20.356756 (+ 421us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:20.364808 (+ 8052us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:20.364898 (+ 90us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:20.364974 (+ 76us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:20.357706 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:20.357830 (+ 124us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:20.357837 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:20.357839 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:20.357932 (+ 93us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:20.357946 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:20.357951 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:20.358028 (+ 77us) tablet.cc:701] Row locks acquired
0505 07:54:20.358030 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:20.358065 (+ 35us) write_op.cc:270] Start()
0505 07:54:20.358099 (+ 34us) write_op.cc:276] Timestamp: P: 1777967660358004 usec, L: 0
0505 07:54:20.358103 (+ 4us) op_driver.cc:348] REPLICATION: starting
0505 07:54:20.358287 (+ 184us) log.cc:844] Serialized 433 byte log entry
0505 07:54:20.362204 (+ 3917us) op_driver.cc:464] REPLICATION: finished
0505 07:54:20.364108 (+ 1904us) write_op.cc:301] APPLY: starting
0505 07:54:20.364124 (+ 16us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:20.364161 (+ 37us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:20.364205 (+ 44us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:20.364329 (+ 124us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:20.364360 (+ 31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:20.364367 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:20.364438 (+ 71us) log.cc:844] Serialized 29 byte log entry
0505 07:54:20.364499 (+ 61us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.364517 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:20.364522 (+ 5us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:20.364626 (+ 104us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.364629 (+ 3us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1843,"apply.run_cpu_time_us":514,"apply.run_wall_time_us":515,"num_ops":2,"prepare.queue_time_us":465,"prepare.run_cpu_time_us":712,"prepare.run_wall_time_us":712,"raft.queue_time_us":949,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":223,"replication_time_us":4067}]]}
I20260505 07:54:20.366322 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_53389d064ac94806a9f9b3e767c99ed8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260505 07:54:20.367107 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_53389d064ac94806a9f9b3e767c99ed8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:20.352926 745 tablet_service.cc:1558] Processing DeleteTablet for tablet 75de7522fa534140822ead8be4dc2b2d with delete_type TABLET_DATA_DELETED (replaced by tablet 8e1e4541c93342aca228008a6894522e) from {username='slave'} at 127.0.0.1:55776
W20260505 07:54:20.372021 1317 consensus_peers.cc:597] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311 -> Peer 28d3eda04f9c47ae976d19c218b6dcc5 (127.0.105.60:35401): Couldn't send request to peer 28d3eda04f9c47ae976d19c218b6dcc5. Status: Network error: Client connection negotiation failed: client connection to 127.0.105.60:35401: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260505 07:54:20.379136 1181 tablet_bootstrap.cc:492] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.379319 1181 ts_tablet_manager.cc:1403] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.050s user 0.003s sys 0.000s
I20260505 07:54:20.380080 1181 raft_consensus.cc:359] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.380291 1181 raft_consensus.cc:385] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.380390 1181 raft_consensus.cc:740] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.380620 1181 consensus_queue.cc:260] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.381309 1181 ts_tablet_manager.cc:1434] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260505 07:54:20.382587 1181 tablet_bootstrap.cc:492] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.382725 1355 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 278) took 1741 ms. Trace:
I20260505 07:54:20.382843 3144 raft_consensus.cc:493] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.383034 3144 raft_consensus.cc:515] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.382839 1355 rpcz_store.cc:276] 0505 07:54:18.640856 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:20.366138 (+1725282us) service_pool.cc:224] Handling call
0505 07:54:20.367246 (+ 1108us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:20.367754 (+ 508us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:20.382479 (+ 14725us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:20.382574 (+ 95us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:20.382669 (+ 95us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:20.370492 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:20.370600 (+ 108us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:20.370604 (+ 4us) write_op.cc:435] Acquired schema lock
0505 07:54:20.370606 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:20.370674 (+ 68us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:20.370686 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:20.370692 (+ 6us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:20.370787 (+ 95us) tablet.cc:701] Row locks acquired
0505 07:54:20.370790 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:20.370822 (+ 32us) write_op.cc:270] Start()
0505 07:54:20.370850 (+ 28us) write_op.cc:276] Timestamp: P: 1777967660370759 usec, L: 0
0505 07:54:20.370853 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:20.370988 (+ 135us) log.cc:844] Serialized 433 byte log entry
0505 07:54:20.380590 (+ 9602us) op_driver.cc:464] REPLICATION: finished
0505 07:54:20.380699 (+ 109us) write_op.cc:301] APPLY: starting
0505 07:54:20.380712 (+ 13us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:20.380749 (+ 37us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:20.380792 (+ 43us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:20.380881 (+ 89us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:20.380910 (+ 29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:20.380917 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:20.380981 (+ 64us) log.cc:844] Serialized 29 byte log entry
0505 07:54:20.381046 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.381066 (+ 20us) write_op.cc:454] Released schema lock
0505 07:54:20.381072 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:20.381137 (+ 65us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.381141 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":72,"apply.run_cpu_time_us":429,"apply.run_wall_time_us":428,"num_ops":2,"prepare.queue_time_us":193,"prepare.run_cpu_time_us":586,"prepare.run_wall_time_us":585,"raft.queue_time_us":7040,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":4317,"replication_time_us":9714}]]}
I20260505 07:54:20.383490 1355 master_service.cc:954] ReplaceTablet: received request to replace tablet 39eeecbfd10e4df4a89c8598aefcfd88 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:20.383770 3144 leader_election.cc:290] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:20.385052 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839b963b04b24cf8af8d358a69af4f39" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.385072 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839b963b04b24cf8af8d358a69af4f39" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.385301 1181 tablet_bootstrap.cc:654] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:20.385747 856 leader_election.cc:343] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:20.386315 854 leader_election.cc:343] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.386466 854 leader_election.cc:304] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.386988 3144 raft_consensus.cc:2749] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.376180 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 236) took 7976 ms. Trace:
I20260505 07:54:20.388094 3144 raft_consensus.cc:493] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.388232 3144 raft_consensus.cc:515] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.388888 3144 leader_election.cc:290] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:20.389762 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b5840ac386e47d6a1229015549f9d92" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.389822 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b5840ac386e47d6a1229015549f9d92" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.389946 2268 ts_tablet_manager.cc:1929] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.390295 2268 log.cc:1199] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/ba2494780bdd4b6dae6fbdf71125da88
W20260505 07:54:20.390254 856 leader_election.cc:343] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:20.390730 854 leader_election.cc:343] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.390852 854 leader_election.cc:304] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.391240 3144 raft_consensus.cc:2749] T 9b5840ac386e47d6a1229015549f9d92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.391546 2268 ts_tablet_manager.cc:1950] T ba2494780bdd4b6dae6fbdf71125da88 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:20.387360 2276 rpcz_store.cc:276] 0505 07:54:12.399767 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.399839 (+ 72us) service_pool.cc:224] Handling call
0505 07:54:20.332131 (+7932292us) ts_tablet_manager.cc:1151] Deleting tablet f840d5f58161468385d5c4f116b78c3d
0505 07:54:20.332141 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.338630 (+ 6489us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.342532 (+ 3902us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.376150 (+ 33618us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7932098}
I20260505 07:54:20.391913 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet f840d5f58161468385d5c4f116b78c3d (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]) successfully deleted
I20260505 07:54:20.391999 2276 tablet_replica.cc:333] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.392248 2276 raft_consensus.cc:2243] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.392445 2276 raft_consensus.cc:2272] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.393103 2276 ts_tablet_manager.cc:1916] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.393793 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 363) took 4161 ms. Trace:
I20260505 07:54:20.400363 2447 ts_tablet_manager.cc:1929] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.400538 2447 log.cc:1199] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/b256d072933b4566b5587224274deb66
I20260505 07:54:20.400636 2268 rpcz_store.cc:276] 0505 07:54:16.232818 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.232858 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:20.306712 (+4073854us) ts_tablet_manager.cc:1151] Deleting tablet ba2494780bdd4b6dae6fbdf71125da88
0505 07:54:20.306723 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.340256 (+ 33533us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.387149 (+ 46893us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.393777 (+ 6628us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4064727}
I20260505 07:54:20.401314 2447 ts_tablet_manager.cc:1950] T b256d072933b4566b5587224274deb66 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.401667 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet ba2494780bdd4b6dae6fbdf71125da88 (table default.Table_063cc9f08ab54458b9e277ca65091b11 [id=4f094780dd764a01bb6ba8c7b08d6ef0]) successfully deleted
I20260505 07:54:20.400041 1181 tablet_bootstrap.cc:492] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.403112 1181 ts_tablet_manager.cc:1403] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.000s
I20260505 07:54:20.404134 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 371) took 3812 ms. Trace:
I20260505 07:54:20.404383 2268 rpcz_store.cc:276] 0505 07:54:16.591330 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.591368 (+ 38us) service_pool.cc:224] Handling call
0505 07:54:20.404018 (+3812650us) ts_tablet_manager.cc:1151] Deleting tablet bf0ce3e6c00541569d15c1b6ef347579
0505 07:54:20.404036 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.404119 (+ 83us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3812481}
W20260505 07:54:20.409363 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet bf0ce3e6c00541569d15c1b6ef347579 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bf0ce3e6c00541569d15c1b6ef347579 already in progress: creating tablet
I20260505 07:54:20.409453 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 380) took 3532 ms. Trace:
I20260505 07:54:20.408934 1181 raft_consensus.cc:359] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.409514 2268 rpcz_store.cc:276] 0505 07:54:16.877009 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.877046 (+ 37us) service_pool.cc:224] Handling call
0505 07:54:20.409373 (+3532327us) ts_tablet_manager.cc:1151] Deleting tablet c1a61c1d03d544c8aa01537ead97f83c
0505 07:54:20.409384 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.409439 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3532178}
I20260505 07:54:20.409694 1181 raft_consensus.cc:385] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.411478 1181 raft_consensus.cc:740] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.412467 1181 consensus_queue.cc:260] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:54:20.412887 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet c1a61c1d03d544c8aa01537ead97f83c because the tablet was not found. No further retry: Not found: Tablet not found: c1a61c1d03d544c8aa01537ead97f83c
I20260505 07:54:20.413264 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 245) took 9750 ms. Trace:
I20260505 07:54:20.413383 2447 rpcz_store.cc:276] 0505 07:54:10.662943 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.663993 (+ 1050us) service_pool.cc:224] Handling call
0505 07:54:20.324055 (+9660062us) ts_tablet_manager.cc:1151] Deleting tablet b256d072933b4566b5587224274deb66
0505 07:54:20.324065 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.375459 (+ 51394us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.400272 (+ 24813us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.413230 (+ 12958us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9659932}
I20260505 07:54:20.413807 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet b256d072933b4566b5587224274deb66 (table default.Table_6cb216f92da74a78acd72a6132b94c1a [id=c0306ceddf024a1589f83f390b38c602]) successfully deleted
I20260505 07:54:20.412959 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 381) took 3535 ms. Trace:
I20260505 07:54:20.413978 2268 rpcz_store.cc:276] 0505 07:54:16.877351 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.877390 (+ 39us) service_pool.cc:224] Handling call
0505 07:54:20.412882 (+3535492us) ts_tablet_manager.cc:1151] Deleting tablet ff7ed926a3ff4e7993864de734ca3c50
0505 07:54:20.412892 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.412945 (+ 53us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3535342}
W20260505 07:54:20.415354 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet ff7ed926a3ff4e7993864de734ca3c50 because the tablet was not found. No further retry: Not found: Tablet not found: ff7ed926a3ff4e7993864de734ca3c50
I20260505 07:54:20.415735 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 379) took 3538 ms. Trace:
I20260505 07:54:20.416097 2447 tablet_replica.cc:333] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.416234 2268 rpcz_store.cc:276] 0505 07:54:16.876930 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.877435 (+ 505us) service_pool.cc:224] Handling call
0505 07:54:20.415658 (+3538223us) ts_tablet_manager.cc:1151] Deleting tablet 7001482d90364dbdb8b9c5bfb98e1cd4
0505 07:54:20.415667 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.415720 (+ 53us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3538097}
I20260505 07:54:20.414947 1181 ts_tablet_manager.cc:1434] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.012s user 0.002s sys 0.000s
W20260505 07:54:20.416838 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 because the tablet was not found. No further retry: Not found: Tablet not found: 7001482d90364dbdb8b9c5bfb98e1cd4
I20260505 07:54:20.417652 1181 tablet_bootstrap.cc:492] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.417928 2447 raft_consensus.cc:2243] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.419909 2447 raft_consensus.cc:2272] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.418238 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 382) took 3540 ms. Trace:
I20260505 07:54:20.420100 2268 rpcz_store.cc:276] 0505 07:54:16.877698 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.877739 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:20.418156 (+3540417us) ts_tablet_manager.cc:1151] Deleting tablet f121fdd001ec4ea38ccd24a6afd9e51c
0505 07:54:20.418167 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.418222 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3540266}
I20260505 07:54:20.420328 3144 raft_consensus.cc:493] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.420466 3144 raft_consensus.cc:515] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.420828 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 383) took 3543 ms. Trace:
I20260505 07:54:20.420953 3144 leader_election.cc:290] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:20.420987 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f121fdd001ec4ea38ccd24a6afd9e51c because the tablet was not found. No further retry: Not found: Tablet not found: f121fdd001ec4ea38ccd24a6afd9e51c
I20260505 07:54:20.421656 2447 ts_tablet_manager.cc:1916] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.421588 1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5006965919824adbbb23d744fb9ff335" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.422127 2268 rpcz_store.cc:276] 0505 07:54:16.877789 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.877828 (+ 39us) service_pool.cc:224] Handling call
0505 07:54:20.420755 (+3542927us) ts_tablet_manager.cc:1151] Deleting tablet 93d878ae0ddb42319560f74e0a490321
0505 07:54:20.420763 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.420812 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3542758}
W20260505 07:54:20.422250 856 leader_election.cc:343] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.422816 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 384) took 3544 ms. Trace:
I20260505 07:54:20.423141 2268 rpcz_store.cc:276] 0505 07:54:16.878161 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.878196 (+ 35us) service_pool.cc:224] Handling call
0505 07:54:20.422733 (+3544537us) ts_tablet_manager.cc:1151] Deleting tablet 0459f2d45a2d44f1a73e0a0fff53866c
0505 07:54:20.422744 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.422799 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3544398}
W20260505 07:54:20.423821 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 93d878ae0ddb42319560f74e0a490321 because the tablet was not found. No further retry: Not found: Tablet not found: 93d878ae0ddb42319560f74e0a490321
I20260505 07:54:20.421627 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5006965919824adbbb23d744fb9ff335" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:20.424357 854 leader_election.cc:343] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.424453 854 leader_election.cc:304] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.424908 3144 raft_consensus.cc:2749] T 5006965919824adbbb23d744fb9ff335 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260505 07:54:20.427887 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 0459f2d45a2d44f1a73e0a0fff53866c because the tablet was not found. No further retry: Not found: Tablet not found: 0459f2d45a2d44f1a73e0a0fff53866c
I20260505 07:54:20.427985 2268 tablet_replica.cc:333] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:20.428195 2268 raft_consensus.cc:2243] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.429046 2268 raft_consensus.cc:2272] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.428825 1181 tablet_bootstrap.cc:654] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.430507 2268 ts_tablet_manager.cc:1916] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.432348 1172 tablet_bootstrap.cc:492] T 3fece4034c3a42169fae0c9f41dce0bb P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.432464 1172 ts_tablet_manager.cc:1403] T 3fece4034c3a42169fae0c9f41dce0bb P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.096s user 0.038s sys 0.042s
I20260505 07:54:20.433154 1172 raft_consensus.cc:359] T 3fece4034c3a42169fae0c9f41dce0bb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.433295 1172 raft_consensus.cc:385] T 3fece4034c3a42169fae0c9f41dce0bb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.433342 1172 raft_consensus.cc:740] T 3fece4034c3a42169fae0c9f41dce0bb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.433532 1172 consensus_queue.cc:260] T 3fece4034c3a42169fae0c9f41dce0bb P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.433954 1172 ts_tablet_manager.cc:1434] T 3fece4034c3a42169fae0c9f41dce0bb P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260505 07:54:20.434494 1172 tablet_bootstrap.cc:492] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:20.436157 1172 tablet_bootstrap.cc:654] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.436456 1181 tablet_bootstrap.cc:492] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.436775 1181 ts_tablet_manager.cc:1403] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260505 07:54:20.437783 1181 raft_consensus.cc:359] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.448436 1181 raft_consensus.cc:385] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.448731 1181 raft_consensus.cc:740] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.449399 1181 consensus_queue.cc:260] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.450083 2276 ts_tablet_manager.cc:1929] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.450217 2276 log.cc:1199] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/b5d113a872f9400c8f28d0239ca42377
I20260505 07:54:20.450682 2276 ts_tablet_manager.cc:1950] T b5d113a872f9400c8f28d0239ca42377 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.451481 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 237) took 7688 ms. Trace:
I20260505 07:54:20.451973 1181 ts_tablet_manager.cc:1434] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20260505 07:54:20.456457 1181 tablet_bootstrap.cc:492] T 3ba4533c78974853b48526c465e38941 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.456485 1176 tablet_bootstrap.cc:492] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:20.456653 1176 ts_tablet_manager.cc:1403] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.110s user 0.000s sys 0.011s
I20260505 07:54:20.458027 1176 raft_consensus.cc:359] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.458230 1176 raft_consensus.cc:385] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.458326 1176 raft_consensus.cc:740] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.458575 1176 consensus_queue.cc:260] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.459988 1181 tablet_bootstrap.cc:654] T 3ba4533c78974853b48526c465e38941 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.460307 2447 ts_tablet_manager.cc:1929] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.460486 2447 log.cc:1199] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/17b4e886bebc4ab6a3c95290d18b4229
I20260505 07:54:20.461561 2447 ts_tablet_manager.cc:1950] T 17b4e886bebc4ab6a3c95290d18b4229 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.462348 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 246) took 9722 ms. Trace:
I20260505 07:54:20.463394 2268 ts_tablet_manager.cc:1929] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.463690 2447 rpcz_store.cc:276] 0505 07:54:10.740333 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.740381 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:20.416018 (+9675637us) ts_tablet_manager.cc:1151] Deleting tablet 17b4e886bebc4ab6a3c95290d18b4229
0505 07:54:20.416037 (+ 19us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.444660 (+ 28623us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.460232 (+ 15572us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.462288 (+ 2056us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9675467}
I20260505 07:54:20.464329 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 247) took 9711 ms. Trace:
I20260505 07:54:20.464493 2268 log.cc:1199] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/05351965bdba4ccca8d13223cd505948
I20260505 07:54:20.464865 2268 ts_tablet_manager.cc:1950] T 05351965bdba4ccca8d13223cd505948 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:20.464977 2447 rpcz_store.cc:276] 0505 07:54:10.753192 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.753241 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:20.464180 (+9710939us) ts_tablet_manager.cc:1151] Deleting tablet 91d57f5bb6e847e6b099134d1382b730
0505 07:54:20.464190 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.464281 (+ 91us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9710788}
I20260505 07:54:20.465469 2447 tablet_replica.cc:333] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.466236 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 385) took 3587 ms. Trace:
I20260505 07:54:20.466514 2447 raft_consensus.cc:2243] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.466909 2447 raft_consensus.cc:2272] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.467155 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 17b4e886bebc4ab6a3c95290d18b4229 (table default.Table_dcd2d0ba71434f6bb31d92683f67408c [id=a6476fbd5ce8492e8d4e54238c5c019f]) successfully deleted
I20260505 07:54:20.467398 2268 rpcz_store.cc:276] 0505 07:54:16.878536 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.878581 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:20.427942 (+3549361us) ts_tablet_manager.cc:1151] Deleting tablet 05351965bdba4ccca8d13223cd505948
0505 07:54:20.427952 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.453062 (+ 25110us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.463345 (+ 10283us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.466217 (+ 2872us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3549210}
W20260505 07:54:20.467507 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 91d57f5bb6e847e6b099134d1382b730 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 91d57f5bb6e847e6b099134d1382b730 already in progress: creating tablet
I20260505 07:54:20.467906 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 05351965bdba4ccca8d13223cd505948 (table default.Table_54f371d41b5e4a54b08bb82a834ac0af [id=abda2c3e5e9d4c64b0d08510cb3d8aed]) successfully deleted
I20260505 07:54:20.469170 1176 ts_tablet_manager.cc:1434] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.012s user 0.000s sys 0.002s
I20260505 07:54:20.469662 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 386) took 3589 ms. Trace:
I20260505 07:54:20.469923 2447 ts_tablet_manager.cc:1916] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.470046 2268 rpcz_store.cc:276] 0505 07:54:16.878847 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.878895 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:20.467903 (+3589008us) ts_tablet_manager.cc:1151] Deleting tablet 4f3e076892234c1494b1a5c13cbfd2e2
0505 07:54:20.467913 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.467978 (+ 65us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3588838}
W20260505 07:54:20.470475 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 4f3e076892234c1494b1a5c13cbfd2e2 because the tablet was not found. No further retry: Not found: Tablet not found: 4f3e076892234c1494b1a5c13cbfd2e2
I20260505 07:54:20.470759 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 387) took 3591 ms. Trace:
I20260505 07:54:20.470970 1176 tablet_bootstrap.cc:492] T ad5c570d4861460d8ab0c8ff8a7a3a03 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:20.471105 2268 rpcz_store.cc:276] 0505 07:54:16.879201 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.879247 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:20.470675 (+3591428us) ts_tablet_manager.cc:1151] Deleting tablet a9494c3b6a99487596a030856b371b68
0505 07:54:20.470685 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.470742 (+ 57us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3591297}
I20260505 07:54:20.472101 2276 rpcz_store.cc:276] 0505 07:54:12.762630 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.762679 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:20.391941 (+7629262us) ts_tablet_manager.cc:1151] Deleting tablet b5d113a872f9400c8f28d0239ca42377
0505 07:54:20.391953 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.401291 (+ 9338us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.450019 (+ 48728us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.451462 (+ 1443us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7629074}
I20260505 07:54:20.471400 3173 raft_consensus.cc:493] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.472569 3173 raft_consensus.cc:515] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.472716 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet b5d113a872f9400c8f28d0239ca42377 (table default.Table_e9f1ff58ca6d41f19d05c458dc529941 [id=7579207045ec4220b966af9016a8f050]) successfully deleted
I20260505 07:54:20.473152 3173 leader_election.cc:290] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:20.473675 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f5b72f58a24db0b58c15b47227bc60" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:20.474558 988 leader_election.cc:343] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.475047 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f5b72f58a24db0b58c15b47227bc60" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.475211 907 raft_consensus.cc:2468] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:20.475663 991 leader_election.cc:304] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.476226 3202 raft_consensus.cc:2804] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:20.476297 3202 raft_consensus.cc:493] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:20.476361 3202 raft_consensus.cc:3060] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.478886 3202 raft_consensus.cc:515] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.479449 3202 leader_election.cc:290] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:20.479812 2276 tablet_replica.cc:333] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.480082 2276 raft_consensus.cc:2243] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.480260 2276 raft_consensus.cc:2272] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.480903 2276 ts_tablet_manager.cc:1916] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.481729 778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f5b72f58a24db0b58c15b47227bc60" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:20.482167 988 leader_election.cc:343] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.483515 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2f5b72f58a24db0b58c15b47227bc60" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:20.483708 907 raft_consensus.cc:3060] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.486305 907 raft_consensus.cc:2468] T f2f5b72f58a24db0b58c15b47227bc60 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:20.488616 3296 consensus_queue.cc:1048] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.490461 991 leader_election.cc:304] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.490819 3203 raft_consensus.cc:2804] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:20.490934 3203 raft_consensus.cc:697] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:20.491173 3203 consensus_queue.cc:237] T f2f5b72f58a24db0b58c15b47227bc60 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.496459 1176 tablet_bootstrap.cc:654] T ad5c570d4861460d8ab0c8ff8a7a3a03 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:20.502677 718 consensus_peers.cc:597] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e82e9ae417784b7e83f4843b129d0580. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:20.503204 1172 tablet_bootstrap.cc:492] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.503337 1172 ts_tablet_manager.cc:1403] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.069s user 0.014s sys 0.034s
I20260505 07:54:20.504002 1172 raft_consensus.cc:359] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.504164 1172 raft_consensus.cc:385] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.504221 1172 raft_consensus.cc:740] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.505774 1172 consensus_queue.cc:260] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.511765 1172 ts_tablet_manager.cc:1434] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20260505 07:54:20.511977 3144 consensus_queue.cc:1048] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.512579 1172 tablet_bootstrap.cc:492] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:20.514642 1172 tablet_bootstrap.cc:654] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.515967 2276 ts_tablet_manager.cc:1929] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.516094 2276 log.cc:1199] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/c4dd3ece5cb34ceb82feeb67e75da0b5
I20260505 07:54:20.516494 2276 ts_tablet_manager.cc:1950] T c4dd3ece5cb34ceb82feeb67e75da0b5 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.517436 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 238) took 7705 ms. Trace:
I20260505 07:54:20.517539 2276 rpcz_store.cc:276] 0505 07:54:12.812396 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:12.812444 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:20.479743 (+7667299us) ts_tablet_manager.cc:1151] Deleting tablet c4dd3ece5cb34ceb82feeb67e75da0b5
0505 07:54:20.479757 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.485728 (+ 5971us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.515882 (+ 30154us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.517418 (+ 1536us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7667141}
I20260505 07:54:20.518083 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet c4dd3ece5cb34ceb82feeb67e75da0b5 (table default.Table_d4c7087457154e89b20abe713355e422 [id=7099b9d0095f4bf880cff569eeeb5d32]) successfully deleted
I20260505 07:54:20.527385 2276 tablet_replica.cc:333] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.531745 2276 raft_consensus.cc:2243] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.532025 2276 raft_consensus.cc:2272] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.533149 3144 raft_consensus.cc:493] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.533299 3144 raft_consensus.cc:515] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.534005 3144 leader_election.cc:290] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.534610 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0ce3e6c00541569d15c1b6ef347579" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.534719 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf0ce3e6c00541569d15c1b6ef347579" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:20.535302 854 leader_election.cc:343] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:20.535632 856 leader_election.cc:343] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.535681 2276 ts_tablet_manager.cc:1916] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.535754 856 leader_election.cc:304] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.536198 3144 raft_consensus.cc:2749] T bf0ce3e6c00541569d15c1b6ef347579 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.542780 1181 tablet_bootstrap.cc:492] T 3ba4533c78974853b48526c465e38941 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.542979 1181 ts_tablet_manager.cc:1403] T 3ba4533c78974853b48526c465e38941 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.087s user 0.003s sys 0.000s
I20260505 07:54:20.550442 3403 tablet.cc:1724] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d2799f5c6b4f4336b976ca9bbb458533: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:20.551775 3397 tablet.cc:1724] T 139d5a2ab9594f6ea6042c1c4ba8a46a P eac7e1e8caba43cdb4df80cbb80e891c: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:20.552191 3177 consensus_queue.cc:1048] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.562208 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 388) took 3682 ms. Trace:
W20260505 07:54:20.562232 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet a9494c3b6a99487596a030856b371b68 because the tablet was not found. No further retry: Not found: Tablet not found: a9494c3b6a99487596a030856b371b68
I20260505 07:54:20.562876 2268 rpcz_store.cc:276] 0505 07:54:16.879472 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.879504 (+ 32us) service_pool.cc:224] Handling call
0505 07:54:20.562111 (+3682607us) ts_tablet_manager.cc:1151] Deleting tablet 8b406c4c6ba741bc85eef54c6f576459
0505 07:54:20.562121 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.562187 (+ 66us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3682475}
W20260505 07:54:20.563814 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8b406c4c6ba741bc85eef54c6f576459 because the tablet was not found. No further retry: Not found: Tablet not found: 8b406c4c6ba741bc85eef54c6f576459
I20260505 07:54:20.563975 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 389) took 3684 ms. Trace:
I20260505 07:54:20.564569 2268 rpcz_store.cc:276] 0505 07:54:16.879759 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.879802 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:20.563896 (+3684094us) ts_tablet_manager.cc:1151] Deleting tablet 1852f7f4ff594639908b03730b68fe86
0505 07:54:20.563906 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.563959 (+ 53us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3683948}
W20260505 07:54:20.569139 718 consensus_peers.cc:597] T 25072055f5534078adf5b5025614f951 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 25072055f5534078adf5b5025614f951. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:20.569689 1181 raft_consensus.cc:359] T 3ba4533c78974853b48526c465e38941 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:54:20.569985 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 1852f7f4ff594639908b03730b68fe86 because the tablet was not found. No further retry: Not found: Tablet not found: 1852f7f4ff594639908b03730b68fe86
I20260505 07:54:20.570140 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 390) took 3690 ms. Trace:
I20260505 07:54:20.572284 2268 rpcz_store.cc:276] 0505 07:54:16.880063 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.880097 (+ 34us) service_pool.cc:224] Handling call
0505 07:54:20.570057 (+3689960us) ts_tablet_manager.cc:1151] Deleting tablet 02716f62aa354d8f839266661b7f05ea
0505 07:54:20.570066 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.570124 (+ 58us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3689812}
W20260505 07:54:20.573176 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 02716f62aa354d8f839266661b7f05ea because the tablet was not found. No further retry: Not found: Tablet not found: 02716f62aa354d8f839266661b7f05ea
I20260505 07:54:20.573316 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 391) took 3692 ms. Trace:
I20260505 07:54:20.576934 1176 tablet_bootstrap.cc:492] T ad5c570d4861460d8ab0c8ff8a7a3a03 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:20.580653 2268 rpcz_store.cc:276] 0505 07:54:16.880367 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.880411 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:20.573218 (+3692807us) ts_tablet_manager.cc:1151] Deleting tablet 09b46ff0101a4e529077b9e30492ace7
0505 07:54:20.573227 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.573282 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3692676}
W20260505 07:54:20.581166 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 09b46ff0101a4e529077b9e30492ace7 because the tablet was not found. No further retry: Not found: Tablet not found: 09b46ff0101a4e529077b9e30492ace7
I20260505 07:54:20.581477 2268 tablet_replica.cc:333] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:20.581704 2268 raft_consensus.cc:2243] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.581905 2268 raft_consensus.cc:2272] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.582441 2268 ts_tablet_manager.cc:1916] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.569921 1181 raft_consensus.cc:385] T 3ba4533c78974853b48526c465e38941 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.583895 1181 raft_consensus.cc:740] T 3ba4533c78974853b48526c465e38941 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.584205 1181 consensus_queue.cc:260] T 3ba4533c78974853b48526c465e38941 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.584920 1181 ts_tablet_manager.cc:1434] T 3ba4533c78974853b48526c465e38941 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.042s user 0.003s sys 0.000s
I20260505 07:54:20.585307 1181 tablet_bootstrap.cc:492] T 5006965919824adbbb23d744fb9ff335 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.590826 2447 ts_tablet_manager.cc:1929] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.594408 2268 ts_tablet_manager.cc:1929] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.594594 2447 log.cc:1199] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/4f3e076892234c1494b1a5c13cbfd2e2
I20260505 07:54:20.595041 2447 ts_tablet_manager.cc:1950] T 4f3e076892234c1494b1a5c13cbfd2e2 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.595857 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 250) took 9621 ms. Trace:
I20260505 07:54:20.596031 2447 rpcz_store.cc:276] 0505 07:54:10.974860 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.974965 (+ 105us) service_pool.cc:224] Handling call
0505 07:54:20.465394 (+9490429us) ts_tablet_manager.cc:1151] Deleting tablet 4f3e076892234c1494b1a5c13cbfd2e2
0505 07:54:20.465404 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.566770 (+101366us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.590734 (+ 23964us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.595821 (+ 5087us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9490260}
I20260505 07:54:20.596473 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 4f3e076892234c1494b1a5c13cbfd2e2 (table default.Table_fe563815e7df4183a9bfe81907c3c33c [id=fa1b1c24909247538d57d1a37ea9513b]) successfully deleted
I20260505 07:54:20.596609 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 251) took 9621 ms. Trace:
I20260505 07:54:20.596716 2447 rpcz_store.cc:276] 0505 07:54:10.975025 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.975161 (+ 136us) service_pool.cc:224] Handling call
0505 07:54:20.596497 (+9621336us) ts_tablet_manager.cc:1151] Deleting tablet a9494c3b6a99487596a030856b371b68
0505 07:54:20.596509 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.596579 (+ 70us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9621193}
W20260505 07:54:20.597088 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet a9494c3b6a99487596a030856b371b68 because the tablet was not found. No further retry: Not found: Tablet not found: a9494c3b6a99487596a030856b371b68
I20260505 07:54:20.597218 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 252) took 9621 ms. Trace:
I20260505 07:54:20.597323 2447 rpcz_store.cc:276] 0505 07:54:10.976134 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.976233 (+ 99us) service_pool.cc:224] Handling call
0505 07:54:20.597099 (+9620866us) ts_tablet_manager.cc:1151] Deleting tablet ae829daad64545e687a9528e5c38b9ed
0505 07:54:20.597110 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.597187 (+ 77us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9620719}
W20260505 07:54:20.597651 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet ae829daad64545e687a9528e5c38b9ed because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ae829daad64545e687a9528e5c38b9ed already in progress: creating tablet
I20260505 07:54:20.597718 2447 tablet_replica.cc:333] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.597934 2447 raft_consensus.cc:2243] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.598282 2447 raft_consensus.cc:2272] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.598820 2447 ts_tablet_manager.cc:1916] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.578945 1176 ts_tablet_manager.cc:1403] T ad5c570d4861460d8ab0c8ff8a7a3a03 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.108s user 0.005s sys 0.004s
I20260505 07:54:20.602519 2276 ts_tablet_manager.cc:1929] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:20.602658 2276 log.cc:1199] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/571257bbb7e94315b60d84d6aa4917b1
I20260505 07:54:20.603132 2276 ts_tablet_manager.cc:1950] T 571257bbb7e94315b60d84d6aa4917b1 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.603482 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 239) took 7178 ms. Trace:
I20260505 07:54:20.604498 2276 rpcz_store.cc:276] 0505 07:54:13.425545 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.425615 (+ 70us) service_pool.cc:224] Handling call
0505 07:54:20.527324 (+7101709us) ts_tablet_manager.cc:1151] Deleting tablet 571257bbb7e94315b60d84d6aa4917b1
0505 07:54:20.527337 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.593980 (+ 66643us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.602458 (+ 8478us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.603463 (+ 1005us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7101474}
I20260505 07:54:20.604436 1172 tablet_bootstrap.cc:492] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.604892 1172 ts_tablet_manager.cc:1403] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.093s user 0.038s sys 0.044s
I20260505 07:54:20.604899 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 571257bbb7e94315b60d84d6aa4917b1 (table default.Table_4ab0ac01b8bf49ef96683dca41dfadd4 [id=6cb8bc8043474bad9f4278b7eef7c71a]) successfully deleted
I20260505 07:54:20.605525 1172 raft_consensus.cc:359] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.605677 1172 raft_consensus.cc:385] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.605734 1172 raft_consensus.cc:740] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.605924 1172 consensus_queue.cc:260] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.606343 1172 ts_tablet_manager.cc:1434] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260505 07:54:20.606571 1172 tablet_bootstrap.cc:492] T 93b546377fd84568aa0f1bf5271b043f P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:20.609088 2276 tablet_replica.cc:333] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.609251 2276 raft_consensus.cc:2243] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.609393 2276 raft_consensus.cc:2272] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.609898 1172 tablet_bootstrap.cc:654] T 93b546377fd84568aa0f1bf5271b043f P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.611304 2276 ts_tablet_manager.cc:1916] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.612274 2268 log.cc:1199] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/e5eaa91ecc1f43d18b313b6be4cc211e
I20260505 07:54:20.612635 2268 ts_tablet_manager.cc:1950] T e5eaa91ecc1f43d18b313b6be4cc211e P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:20.613922 782 raft_consensus.cc:3060] T 9ac029d9a3c540f58f393e79bf7be30e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.609884 1181 tablet_bootstrap.cc:654] T 5006965919824adbbb23d744fb9ff335 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.617918 2276 ts_tablet_manager.cc:1929] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.619351 2276 log.cc:1199] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/4466d0ed977e4803b3b484155095c271
I20260505 07:54:20.619784 2276 ts_tablet_manager.cc:1950] T 4466d0ed977e4803b3b484155095c271 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.622464 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 392) took 3741 ms. Trace:
I20260505 07:54:20.622563 2268 rpcz_store.cc:276] 0505 07:54:16.880642 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.880674 (+ 32us) service_pool.cc:224] Handling call
0505 07:54:20.581419 (+3700745us) ts_tablet_manager.cc:1151] Deleting tablet e5eaa91ecc1f43d18b313b6be4cc211e
0505 07:54:20.581427 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.591437 (+ 10010us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.594342 (+ 2905us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.622424 (+ 28082us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3700631}
I20260505 07:54:20.624590 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet e5eaa91ecc1f43d18b313b6be4cc211e (table default.Table_1ff5286907c6464c8addf47c70eef7ef [id=f608e93cbc64485297c6dc2609a6eb87]) successfully deleted
I20260505 07:54:20.624981 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 240) took 7141 ms. Trace:
I20260505 07:54:20.625103 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 393) took 3744 ms. Trace:
I20260505 07:54:20.625061 2276 rpcz_store.cc:276] 0505 07:54:13.483816 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.484448 (+ 632us) service_pool.cc:224] Handling call
0505 07:54:20.609049 (+7124601us) ts_tablet_manager.cc:1151] Deleting tablet 4466d0ed977e4803b3b484155095c271
0505 07:54:20.609058 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.614021 (+ 4963us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.617876 (+ 3855us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.624962 (+ 7086us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7124339}
I20260505 07:54:20.625255 2268 rpcz_store.cc:276] 0505 07:54:16.880917 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:16.880955 (+ 38us) service_pool.cc:224] Handling call
0505 07:54:20.624931 (+3743976us) ts_tablet_manager.cc:1151] Deleting tablet 397d33e908af4a0bb8dde26d273be01a
0505 07:54:20.624943 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.625031 (+ 88us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3743848}
I20260505 07:54:20.625366 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 241) took 7135 ms. Trace:
I20260505 07:54:20.625424 2276 rpcz_store.cc:276] 0505 07:54:13.490141 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:13.490192 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:20.625272 (+7135080us) ts_tablet_manager.cc:1151] Deleting tablet dfbe4dff192741099be5ab10a96adeb7
0505 07:54:20.625280 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.625354 (+ 74us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":7134903}
I20260505 07:54:20.625499 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 4466d0ed977e4803b3b484155095c271 (table default.Table_1f316191e5124266935010a5cdcc7a18 [id=88b6ba8aaba145c2b3cac834bb9a988e]) successfully deleted
I20260505 07:54:20.625672 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 242) took 6157 ms. Trace:
W20260505 07:54:20.625725 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 397d33e908af4a0bb8dde26d273be01a because the tablet was not found. No further retry: Not found: Tablet not found: 397d33e908af4a0bb8dde26d273be01a
I20260505 07:54:20.625747 2276 rpcz_store.cc:276] 0505 07:54:14.468100 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.468156 (+ 56us) service_pool.cc:224] Handling call
0505 07:54:20.625605 (+6157449us) ts_tablet_manager.cc:1151] Deleting tablet f121fdd001ec4ea38ccd24a6afd9e51c
0505 07:54:20.625613 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.625657 (+ 44us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6157286}
I20260505 07:54:20.625990 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 243) took 6156 ms. Trace:
W20260505 07:54:20.626037 1322 catalog_manager.cc:4982] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet dfbe4dff192741099be5ab10a96adeb7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dfbe4dff192741099be5ab10a96adeb7 already in progress: creating tablet
I20260505 07:54:20.626049 2276 rpcz_store.cc:276] 0505 07:54:14.469704 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.470074 (+ 370us) service_pool.cc:224] Handling call
0505 07:54:20.625920 (+6155846us) ts_tablet_manager.cc:1151] Deleting tablet 6da10815e0dc41a1a07b6786882fffd3
0505 07:54:20.625927 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.625980 (+ 53us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6155756}
I20260505 07:54:20.626128 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 394) took 3744 ms. Trace:
I20260505 07:54:20.626279 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 245) took 6156 ms. Trace:
I20260505 07:54:20.626209 2268 rpcz_store.cc:276] 0505 07:54:16.881205 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.124410 (+243205us) service_pool.cc:224] Handling call
0505 07:54:20.625990 (+3501580us) ts_tablet_manager.cc:1151] Deleting tablet d77ff90ec2ff4b6eb1795d149459c23a
0505 07:54:20.625999 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.626096 (+ 97us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3501344}
I20260505 07:54:20.626338 2276 rpcz_store.cc:276] 0505 07:54:14.469964 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.470012 (+ 48us) service_pool.cc:224] Handling call
0505 07:54:20.626204 (+6156192us) ts_tablet_manager.cc:1151] Deleting tablet 0db162dfad2448e8ac428b3d3fbe17f7
0505 07:54:20.626211 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.626250 (+ 39us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23,"tablet-delete.queue_time_us":6156036}
I20260505 07:54:20.626582 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 244) took 6156 ms. Trace:
W20260505 07:54:20.626571 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet d77ff90ec2ff4b6eb1795d149459c23a because the tablet was not found. No further retry: Not found: Tablet not found: d77ff90ec2ff4b6eb1795d149459c23a
I20260505 07:54:20.626639 2276 rpcz_store.cc:276] 0505 07:54:14.469833 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.470017 (+ 184us) service_pool.cc:224] Handling call
0505 07:54:20.626525 (+6156508us) ts_tablet_manager.cc:1151] Deleting tablet 067c32ecccb44bca8bf9cc3f6c229326
0505 07:54:20.626532 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.626571 (+ 39us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6156243}
I20260505 07:54:20.626832 2276 tablet_replica.cc:333] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
W20260505 07:54:20.626835 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet f121fdd001ec4ea38ccd24a6afd9e51c because the tablet was not found. No further retry: Not found: Tablet not found: f121fdd001ec4ea38ccd24a6afd9e51c
I20260505 07:54:20.627003 2276 raft_consensus.cc:2243] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:20.627033 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 6da10815e0dc41a1a07b6786882fffd3 because the tablet was not found. No further retry: Not found: Tablet not found: 6da10815e0dc41a1a07b6786882fffd3
I20260505 07:54:20.627121 2276 raft_consensus.cc:2272] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.627194 2268 tablet_replica.cc:333] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
W20260505 07:54:20.627254 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 0db162dfad2448e8ac428b3d3fbe17f7 because the tablet was not found. No further retry: Not found: Tablet not found: 0db162dfad2448e8ac428b3d3fbe17f7
I20260505 07:54:20.627415 2268 raft_consensus.cc:2243] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:20.627766 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 067c32ecccb44bca8bf9cc3f6c229326 because the tablet was not found. No further retry: Not found: Tablet not found: 067c32ecccb44bca8bf9cc3f6c229326
I20260505 07:54:20.628715 2268 raft_consensus.cc:2272] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.627538 2276 ts_tablet_manager.cc:1916] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.630386 2268 ts_tablet_manager.cc:1916] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.632501 1181 tablet_bootstrap.cc:492] T 5006965919824adbbb23d744fb9ff335 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.632688 1181 ts_tablet_manager.cc:1403] T 5006965919824adbbb23d744fb9ff335 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.047s user 0.004s sys 0.000s
I20260505 07:54:20.633399 3298 raft_consensus.cc:493] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.633513 3298 raft_consensus.cc:515] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.633838 1181 raft_consensus.cc:359] T 5006965919824adbbb23d744fb9ff335 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.634020 3298 leader_election.cc:290] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.634110 1181 raft_consensus.cc:385] T 5006965919824adbbb23d744fb9ff335 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.634292 1181 raft_consensus.cc:740] T 5006965919824adbbb23d744fb9ff335 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.634316 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c8c6d524f3f44b2a0cc33d46a39d9ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.634636 1181 consensus_queue.cc:260] T 5006965919824adbbb23d744fb9ff335 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:54:20.634756 854 leader_election.cc:343] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.635290 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c8c6d524f3f44b2a0cc33d46a39d9ea" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:20.635888 856 leader_election.cc:343] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.635987 856 leader_election.cc:304] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.636328 3298 raft_consensus.cc:2749] T 4c8c6d524f3f44b2a0cc33d46a39d9ea P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.638657 2276 ts_tablet_manager.cc:1929] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.638753 2276 log.cc:1199] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/d896e290167b4d0081e8e8b831c8ae2a
I20260505 07:54:20.639025 2276 ts_tablet_manager.cc:1950] T d896e290167b4d0081e8e8b831c8ae2a P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.635335 1181 ts_tablet_manager.cc:1434] T 5006965919824adbbb23d744fb9ff335 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:20.639318 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 246) took 6168 ms. Trace:
I20260505 07:54:20.639394 2276 rpcz_store.cc:276] 0505 07:54:14.470781 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.470825 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:20.626798 (+6155973us) ts_tablet_manager.cc:1151] Deleting tablet d896e290167b4d0081e8e8b831c8ae2a
0505 07:54:20.626805 (+ 7us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.635456 (+ 8651us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.638617 (+ 3161us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.639302 (+ 685us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6155827}
I20260505 07:54:20.640228 2447 ts_tablet_manager.cc:1929] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.640388 1181 tablet_bootstrap.cc:492] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.640425 2447 log.cc:1199] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/a9b7c6e9463342a5a73af12da4447f89
I20260505 07:54:20.640971 2447 ts_tablet_manager.cc:1950] T a9b7c6e9463342a5a73af12da4447f89 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.641259 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet d896e290167b4d0081e8e8b831c8ae2a (table default.Table_948a14e895244fb893e72073a38d1e14 [id=85d8a9438bef4555b0ada2eceffefc0d]) successfully deleted
I20260505 07:54:20.642411 3298 consensus_queue.cc:1048] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.642688 1181 tablet_bootstrap.cc:654] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.643188 2276 tablet_replica.cc:333] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.643383 2276 raft_consensus.cc:2243] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.643512 2276 raft_consensus.cc:2272] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.644557 2276 ts_tablet_manager.cc:1916] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.644464 1176 raft_consensus.cc:359] T ad5c570d4861460d8ab0c8ff8a7a3a03 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.644699 1176 raft_consensus.cc:385] T ad5c570d4861460d8ab0c8ff8a7a3a03 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.649207 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 253) took 9672 ms. Trace:
I20260505 07:54:20.650076 2447 rpcz_store.cc:276] 0505 07:54:10.976233 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.976405 (+ 172us) service_pool.cc:224] Handling call
0505 07:54:20.597660 (+9621255us) ts_tablet_manager.cc:1151] Deleting tablet a9b7c6e9463342a5a73af12da4447f89
0505 07:54:20.597670 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.630818 (+ 33148us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.640161 (+ 9343us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.649140 (+ 8979us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9621129}
I20260505 07:54:20.650693 2447 tablet_replica.cc:333] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.651087 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet a9b7c6e9463342a5a73af12da4447f89 (table default.Table_0393e87d0a314ee898b11935e2f6666d [id=ab93bfbabbad4fad893f7a6dd595540f]) successfully deleted
I20260505 07:54:20.651196 2447 raft_consensus.cc:2243] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.651404 2447 raft_consensus.cc:2272] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.652845 2447 ts_tablet_manager.cc:1916] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.654805 2276 ts_tablet_manager.cc:1929] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.654891 2276 log.cc:1199] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/abaa596871bb4a82aefc317932e91d35
I20260505 07:54:20.655174 2276 ts_tablet_manager.cc:1950] T abaa596871bb4a82aefc317932e91d35 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.655411 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 247) took 6184 ms. Trace:
I20260505 07:54:20.644819 1176 raft_consensus.cc:740] T ad5c570d4861460d8ab0c8ff8a7a3a03 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.655781 1176 consensus_queue.cc:260] T ad5c570d4861460d8ab0c8ff8a7a3a03 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.653645 2268 ts_tablet_manager.cc:1929] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.656261 2268 log.cc:1199] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/60772a9697ba417b879dc6025eeb8eec
I20260505 07:54:20.656754 2268 ts_tablet_manager.cc:1950] T 60772a9697ba417b879dc6025eeb8eec P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:20.655479 2276 rpcz_store.cc:276] 0505 07:54:14.471396 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.471436 (+ 40us) service_pool.cc:224] Handling call
0505 07:54:20.643148 (+6171712us) ts_tablet_manager.cc:1151] Deleting tablet abaa596871bb4a82aefc317932e91d35
0505 07:54:20.643157 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.647219 (+ 4062us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.654768 (+ 7549us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.655399 (+ 631us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6171583}
I20260505 07:54:20.658299 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 248) took 6186 ms. Trace:
I20260505 07:54:20.658377 2276 rpcz_store.cc:276] 0505 07:54:14.471979 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.472023 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:20.658223 (+6186200us) ts_tablet_manager.cc:1151] Deleting tablet ff7ed926a3ff4e7993864de734ca3c50
0505 07:54:20.658232 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.658287 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6186081}
I20260505 07:54:20.658813 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet abaa596871bb4a82aefc317932e91d35 (table default.Table_4dfd9e021e5c40c68fb58d5a1c06c36d [id=1fcae33a3a2d48e7b2000cc7b906c7f5]) successfully deleted
I20260505 07:54:20.659101 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 249) took 6185 ms. Trace:
W20260505 07:54:20.659108 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet ff7ed926a3ff4e7993864de734ca3c50 because the tablet was not found. No further retry: Not found: Tablet not found: ff7ed926a3ff4e7993864de734ca3c50
I20260505 07:54:20.659173 2276 rpcz_store.cc:276] 0505 07:54:14.473557 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.473599 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:20.659032 (+6185433us) ts_tablet_manager.cc:1151] Deleting tablet 12501fdf2cc5491cb1a06db490e70810
0505 07:54:20.659040 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.659088 (+ 48us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6185300}
W20260505 07:54:20.659482 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 12501fdf2cc5491cb1a06db490e70810 because the tablet was not found. No further retry: Not found: Tablet not found: 12501fdf2cc5491cb1a06db490e70810
I20260505 07:54:20.659471 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 395) took 3534 ms. Trace:
I20260505 07:54:20.659564 2268 rpcz_store.cc:276] 0505 07:54:17.125036 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.125086 (+ 50us) service_pool.cc:224] Handling call
0505 07:54:20.627133 (+3502047us) ts_tablet_manager.cc:1151] Deleting tablet 60772a9697ba417b879dc6025eeb8eec
0505 07:54:20.627142 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.635986 (+ 8844us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.653578 (+ 17592us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.659430 (+ 5852us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3501894}
I20260505 07:54:20.659835 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 251) took 6184 ms. Trace:
I20260505 07:54:20.659912 2276 rpcz_store.cc:276] 0505 07:54:14.474914 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.474971 (+ 57us) service_pool.cc:224] Handling call
0505 07:54:20.659770 (+6184799us) ts_tablet_manager.cc:1151] Deleting tablet 46274a44a06b4306b5dac43c66c85a80
0505 07:54:20.659779 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.659822 (+ 43us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6184654}
I20260505 07:54:20.660614 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 253) took 6185 ms. Trace:
I20260505 07:54:20.660692 2276 rpcz_store.cc:276] 0505 07:54:14.475147 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.475297 (+ 150us) service_pool.cc:224] Handling call
0505 07:54:20.660546 (+6185249us) ts_tablet_manager.cc:1151] Deleting tablet 78a6470720984761a8606abeafbef992
0505 07:54:20.660554 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.660601 (+ 47us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6185125}
I20260505 07:54:20.661379 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 254) took 6186 ms. Trace:
I20260505 07:54:20.661473 2276 rpcz_store.cc:276] 0505 07:54:14.475230 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.475275 (+ 45us) service_pool.cc:224] Handling call
0505 07:54:20.661311 (+6186036us) ts_tablet_manager.cc:1151] Deleting tablet f1058fcb9fa6457a85489049ca1d9ada
0505 07:54:20.661319 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.661363 (+ 44us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6185774}
I20260505 07:54:20.661888 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 60772a9697ba417b879dc6025eeb8eec (table default.Table_ee5371f299e44106859f3663fe6052b7 [id=a4d53a8c6539419daba67a153292bb0c]) successfully deleted
I20260505 07:54:20.662271 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 396) took 3536 ms. Trace:
I20260505 07:54:20.661866 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 252) took 6186 ms. Trace:
I20260505 07:54:20.662441 2276 rpcz_store.cc:276] 0505 07:54:14.475028 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.483540 (+ 8512us) service_pool.cc:224] Handling call
0505 07:54:20.661795 (+6178255us) ts_tablet_manager.cc:1151] Deleting tablet 4783dd0b2fbe40219852d850fc0d9a82
0505 07:54:20.661803 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.661853 (+ 50us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6178086}
I20260505 07:54:20.662360 1172 tablet_bootstrap.cc:492] T 93b546377fd84568aa0f1bf5271b043f P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.662777 1172 ts_tablet_manager.cc:1403] T 93b546377fd84568aa0f1bf5271b043f P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.056s user 0.011s sys 0.012s
I20260505 07:54:20.662884 2268 rpcz_store.cc:276] 0505 07:54:17.125436 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.125478 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:20.662174 (+3536696us) ts_tablet_manager.cc:1151] Deleting tablet 02e06d8815034d91986cef12964860ea
0505 07:54:20.662184 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.662237 (+ 53us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3536535}
I20260505 07:54:20.663424 1172 raft_consensus.cc:359] T 93b546377fd84568aa0f1bf5271b043f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.663571 1172 raft_consensus.cc:385] T 93b546377fd84568aa0f1bf5271b043f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.663655 1172 raft_consensus.cc:740] T 93b546377fd84568aa0f1bf5271b043f P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.663856 1172 consensus_queue.cc:260] T 93b546377fd84568aa0f1bf5271b043f P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:20.664083 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 02e06d8815034d91986cef12964860ea because the tablet was not found. No further retry: Not found: Tablet not found: 02e06d8815034d91986cef12964860ea
I20260505 07:54:20.664573 1172 ts_tablet_manager.cc:1434] T 93b546377fd84568aa0f1bf5271b043f P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:20.664947 1172 tablet_bootstrap.cc:492] T 1f043cd9042f4e1180d7c0cd90713b03 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:20.669234 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 250) took 6194 ms. Trace:
I20260505 07:54:20.669308 2276 rpcz_store.cc:276] 0505 07:54:14.474783 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.483831 (+ 9048us) service_pool.cc:224] Handling call
0505 07:54:20.664710 (+6180879us) ts_tablet_manager.cc:1151] Deleting tablet 003cd4472b424aacbba8c5f89900e228
0505 07:54:20.669156 (+ 4446us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.669216 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6180748}
I20260505 07:54:20.669703 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 258) took 6180 ms. Trace:
I20260505 07:54:20.669780 2276 rpcz_store.cc:276] 0505 07:54:14.489263 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.489324 (+ 61us) service_pool.cc:224] Handling call
0505 07:54:20.669620 (+6180296us) ts_tablet_manager.cc:1151] Deleting tablet 141665fcd59048bf926acd0287f1853e
0505 07:54:20.669638 (+ 18us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.669689 (+ 51us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6400,"tablet-delete.queue_time_us":6180138}
I20260505 07:54:20.670092 1172 tablet_bootstrap.cc:654] T 1f043cd9042f4e1180d7c0cd90713b03 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.670255 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 257) took 6181 ms. Trace:
I20260505 07:54:20.670432 3203 consensus_queue.cc:1048] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.670459 2276 rpcz_store.cc:276] 0505 07:54:14.489115 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.489359 (+ 244us) service_pool.cc:224] Handling call
0505 07:54:20.670183 (+6180824us) ts_tablet_manager.cc:1151] Deleting tablet 9a06f77255544d7dabb71a4f70ada809
0505 07:54:20.670193 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.670242 (+ 49us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38,"tablet-delete.queue_time_us":6180694}
I20260505 07:54:20.670832 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 256) took 6181 ms. Trace:
I20260505 07:54:20.670900 2276 rpcz_store.cc:276] 0505 07:54:14.489006 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.489600 (+ 594us) service_pool.cc:224] Handling call
0505 07:54:20.670763 (+6181163us) ts_tablet_manager.cc:1151] Deleting tablet a0f2f46d0e7840458cb992ce2a2dbea2
0505 07:54:20.670772 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.670818 (+ 46us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6181037}
I20260505 07:54:20.675153 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 397) took 3549 ms. Trace:
I20260505 07:54:20.675256 2268 rpcz_store.cc:276] 0505 07:54:17.125835 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.125876 (+ 41us) service_pool.cc:224] Handling call
0505 07:54:20.675050 (+3549174us) ts_tablet_manager.cc:1151] Deleting tablet 1ef4f516e64a44fd8cd33674fb808067
0505 07:54:20.675060 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.675120 (+ 60us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3549026}
W20260505 07:54:20.676568 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 46274a44a06b4306b5dac43c66c85a80 because the tablet was not found. No further retry: Not found: Tablet not found: 46274a44a06b4306b5dac43c66c85a80
W20260505 07:54:20.676800 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 78a6470720984761a8606abeafbef992 because the tablet was not found. No further retry: Not found: Tablet not found: 78a6470720984761a8606abeafbef992
W20260505 07:54:20.677080 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet f1058fcb9fa6457a85489049ca1d9ada because the tablet was not found. No further retry: Not found: Tablet not found: f1058fcb9fa6457a85489049ca1d9ada
W20260505 07:54:20.677318 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 4783dd0b2fbe40219852d850fc0d9a82 because the tablet was not found. No further retry: Not found: Tablet not found: 4783dd0b2fbe40219852d850fc0d9a82
W20260505 07:54:20.677534 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 003cd4472b424aacbba8c5f89900e228 because the tablet was not found. No further retry: Not found: Tablet not found: 003cd4472b424aacbba8c5f89900e228
W20260505 07:54:20.677747 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 141665fcd59048bf926acd0287f1853e because the tablet was not found. No further retry: Not found: Tablet not found: 141665fcd59048bf926acd0287f1853e
I20260505 07:54:20.677914 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 398) took 3551 ms. Trace:
I20260505 07:54:20.678001 2268 rpcz_store.cc:276] 0505 07:54:17.126176 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.126223 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:20.675520 (+3549297us) ts_tablet_manager.cc:1151] Deleting tablet 9a06f77255544d7dabb71a4f70ada809
0505 07:54:20.675528 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.677728 (+ 2200us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3549169}
W20260505 07:54:20.678412 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 9a06f77255544d7dabb71a4f70ada809 because the tablet was not found. No further retry: Not found: Tablet not found: 9a06f77255544d7dabb71a4f70ada809
W20260505 07:54:20.678396 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 9a06f77255544d7dabb71a4f70ada809 because the tablet was not found. No further retry: Not found: Tablet not found: 9a06f77255544d7dabb71a4f70ada809
W20260505 07:54:20.678800 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet a0f2f46d0e7840458cb992ce2a2dbea2 because the tablet was not found. No further retry: Not found: Tablet not found: a0f2f46d0e7840458cb992ce2a2dbea2
I20260505 07:54:20.679730 1181 tablet_bootstrap.cc:492] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.679920 1181 ts_tablet_manager.cc:1403] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.040s user 0.004s sys 0.000s
I20260505 07:54:20.680631 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 255) took 6191 ms. Trace:
I20260505 07:54:20.680752 2276 rpcz_store.cc:276] 0505 07:54:14.488886 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.489847 (+ 961us) service_pool.cc:224] Handling call
0505 07:54:20.680556 (+6190709us) ts_tablet_manager.cc:1151] Deleting tablet cac5064f68d24ebe95b5a0b8e10f463e
0505 07:54:20.680565 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.680616 (+ 51us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6190567}
I20260505 07:54:20.681412 1181 raft_consensus.cc:359] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.681792 2447 ts_tablet_manager.cc:1929] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.681872 2276 tablet_replica.cc:333] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.682096 2276 raft_consensus.cc:2243] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.682241 2276 raft_consensus.cc:2272] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.684080 2276 ts_tablet_manager.cc:1916] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.684276 1181 raft_consensus.cc:385] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.684527 1181 raft_consensus.cc:740] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.684950 1181 consensus_queue.cc:260] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.685824 1181 ts_tablet_manager.cc:1434] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260505 07:54:20.686640 1181 tablet_bootstrap.cc:492] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.682621 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 399) took 3556 ms. Trace:
I20260505 07:54:20.687433 2268 rpcz_store.cc:276] 0505 07:54:17.126516 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.126552 (+ 36us) service_pool.cc:224] Handling call
0505 07:54:20.678222 (+3551670us) ts_tablet_manager.cc:1151] Deleting tablet cac5064f68d24ebe95b5a0b8e10f463e
0505 07:54:20.678231 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.682582 (+ 4351us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3551539}
I20260505 07:54:20.687846 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 400) took 3561 ms. Trace:
I20260505 07:54:20.687980 2268 rpcz_store.cc:276] 0505 07:54:17.126837 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.126870 (+ 33us) service_pool.cc:224] Handling call
0505 07:54:20.687736 (+3560866us) ts_tablet_manager.cc:1151] Deleting tablet f1058fcb9fa6457a85489049ca1d9ada
0505 07:54:20.687744 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.687813 (+ 69us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3560727}
I20260505 07:54:20.688346 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 401) took 3561 ms. Trace:
W20260505 07:54:20.688488 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet f1058fcb9fa6457a85489049ca1d9ada because the tablet was not found. No further retry: Not found: Tablet not found: f1058fcb9fa6457a85489049ca1d9ada
I20260505 07:54:20.688535 2268 rpcz_store.cc:276] 0505 07:54:17.127284 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.127326 (+ 42us) service_pool.cc:224] Handling call
0505 07:54:20.688254 (+3560928us) ts_tablet_manager.cc:1151] Deleting tablet bd581a6799a14a8dafb80d411e8936f3
0505 07:54:20.688262 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.688312 (+ 50us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3560787}
I20260505 07:54:20.688835 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 402) took 3561 ms. Trace:
W20260505 07:54:20.689100 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet bd581a6799a14a8dafb80d411e8936f3 because the tablet was not found. No further retry: Not found: Tablet not found: bd581a6799a14a8dafb80d411e8936f3
I20260505 07:54:20.689329 2447 log.cc:1199] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/02e06d8815034d91986cef12964860ea
I20260505 07:54:20.689981 2268 rpcz_store.cc:276] 0505 07:54:17.127632 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.127668 (+ 36us) service_pool.cc:224] Handling call
0505 07:54:20.688725 (+3561057us) ts_tablet_manager.cc:1151] Deleting tablet 7e707143a4e64725aa66db05b24999a6
0505 07:54:20.688733 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.688797 (+ 64us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3560913}
I20260505 07:54:20.693110 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 403) took 3565 ms. Trace:
W20260505 07:54:20.693192 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 7e707143a4e64725aa66db05b24999a6 because the tablet was not found. No further retry: Not found: Tablet not found: 7e707143a4e64725aa66db05b24999a6
I20260505 07:54:20.693480 2268 rpcz_store.cc:276] 0505 07:54:17.127977 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.128016 (+ 39us) service_pool.cc:224] Handling call
0505 07:54:20.693016 (+3565000us) ts_tablet_manager.cc:1151] Deleting tablet 909306e1e5f34b7c8fbc0d86cafce531
0505 07:54:20.693025 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.693071 (+ 46us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3564867}
W20260505 07:54:20.695043 1318 catalog_manager.cc:4969] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 909306e1e5f34b7c8fbc0d86cafce531 because the tablet was not found. No further retry: Not found: Tablet not found: 909306e1e5f34b7c8fbc0d86cafce531
I20260505 07:54:20.695392 2268 tablet_replica.cc:333] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:20.696741 2268 raft_consensus.cc:2243] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.696976 2268 raft_consensus.cc:2272] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.697548 2268 ts_tablet_manager.cc:1916] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.698629 3202 consensus_queue.cc:1048] T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.694653 1181 tablet_bootstrap.cc:654] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.701814 3364 raft_consensus.cc:493] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.701947 3364 raft_consensus.cc:515] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.702044 1172 tablet_bootstrap.cc:492] T 1f043cd9042f4e1180d7c0cd90713b03 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.702153 1172 ts_tablet_manager.cc:1403] T 1f043cd9042f4e1180d7c0cd90713b03 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.037s user 0.000s sys 0.010s
I20260505 07:54:20.702486 3364 leader_election.cc:290] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.702864 1172 raft_consensus.cc:359] T 1f043cd9042f4e1180d7c0cd90713b03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.703040 1172 raft_consensus.cc:385] T 1f043cd9042f4e1180d7c0cd90713b03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.703111 1172 raft_consensus.cc:740] T 1f043cd9042f4e1180d7c0cd90713b03 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.703315 1172 consensus_queue.cc:260] T 1f043cd9042f4e1180d7c0cd90713b03 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.703960 1172 ts_tablet_manager.cc:1434] T 1f043cd9042f4e1180d7c0cd90713b03 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260505 07:54:20.704223 1172 tablet_bootstrap.cc:492] T 0d86424d75b4491d8967ee0c0203b5ca P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:20.705317 1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f00169aad9420b892100acb4256f92" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.705547 1051 raft_consensus.cc:2468] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:20.706063 1172 tablet_bootstrap.cc:654] T 0d86424d75b4491d8967ee0c0203b5ca P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.706223 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f00169aad9420b892100acb4256f92" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.706421 907 raft_consensus.cc:2468] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 0.
I20260505 07:54:20.706507 717 leader_election.cc:304] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:20.706842 3004 raft_consensus.cc:2804] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:20.706907 3004 raft_consensus.cc:493] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:20.706960 3004 raft_consensus.cc:3060] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.707432 3199 raft_consensus.cc:493] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d63a3e535f8e4dd4a88e447253405b77)
I20260505 07:54:20.707577 3199 raft_consensus.cc:515] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } }
I20260505 07:54:20.708034 3199 leader_election.cc:290] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
W20260505 07:54:20.708751 718 leader_election.cc:343] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: b5d113a872f9400c8f28d0239ca42377
I20260505 07:54:20.709599 3004 raft_consensus.cc:515] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.710191 3004 leader_election.cc:290] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.710477 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f00169aad9420b892100acb4256f92" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:20.711611 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f00169aad9420b892100acb4256f92" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:20.711822 1048 raft_consensus.cc:3060] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.710614 907 raft_consensus.cc:3060] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.714249 1048 raft_consensus.cc:2468] T 27f00169aad9420b892100acb4256f92 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:20.714294 907 raft_consensus.cc:2468] T 27f00169aad9420b892100acb4256f92 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac7e1e8caba43cdb4df80cbb80e891c in term 1.
I20260505 07:54:20.714588 718 leader_election.cc:304] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
W20260505 07:54:20.714905 717 leader_election.cc:343] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: b5d113a872f9400c8f28d0239ca42377
I20260505 07:54:20.714999 717 leader_election.cc:304] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:20.715335 3364 raft_consensus.cc:2804] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:20.715426 3364 raft_consensus.cc:697] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Becoming Leader. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Running, Role: LEADER
I20260505 07:54:20.715641 3364 consensus_queue.cc:237] T 27f00169aad9420b892100acb4256f92 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.716513 3199 raft_consensus.cc:2749] T b5d113a872f9400c8f28d0239ca42377 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260505 07:54:20.720453 717 consensus_peers.cc:597] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Couldn't send request to peer d63a3e535f8e4dd4a88e447253405b77. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5eaa91ecc1f43d18b313b6be4cc211e. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:20.723764 1176 ts_tablet_manager.cc:1434] T ad5c570d4861460d8ab0c8ff8a7a3a03 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.122s user 0.002s sys 0.000s
I20260505 07:54:20.724923 3456 tablet.cc:1724] T 139d5a2ab9594f6ea6042c1c4ba8a46a P d63a3e535f8e4dd4a88e447253405b77: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260505 07:54:20.725557 3004 raft_consensus.cc:493] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.725659 3004 raft_consensus.cc:515] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.726102 3004 leader_election.cc:290] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.727030 1048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0d966bcd7f1419cbd090afc185ae78a" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.734535 3298 raft_consensus.cc:493] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.734648 3298 raft_consensus.cc:515] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.735169 3298 leader_election.cc:290] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.735549 1176 tablet_bootstrap.cc:492] T f2f5b72f58a24db0b58c15b47227bc60 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:20.737231 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0fcf1b5d8944a0b3ee0fe4e85f8608" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:20.737641 856 leader_election.cc:343] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.738091 1176 tablet_bootstrap.cc:654] T f2f5b72f58a24db0b58c15b47227bc60 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.739673 3296 consensus_queue.cc:1048] T 68e608ac01ef4a88906e585bd7ee25ac P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.742563 3203 raft_consensus.cc:493] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.742681 3203 raft_consensus.cc:515] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.743268 3203 leader_election.cc:290] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:20.743891 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84faf2e3d91e4cc394a3411c5b6c0bc7" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:20.744374 988 leader_election.cc:343] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.744652 776 raft_consensus.cc:3060] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.747170 2447 ts_tablet_manager.cc:1950] T 02e06d8815034d91986cef12964860ea P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.747458 2268 ts_tablet_manager.cc:1929] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.747764 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84faf2e3d91e4cc394a3411c5b6c0bc7" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.747982 918 raft_consensus.cc:2468] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:20.749015 1355 catalog_manager.cc:5958] T 00000000000000000000000000000000 P 23c57a2c927c4c54bccecfeb3a775311: Sending DeleteTablet for 3 replicas of tablet 39eeecbfd10e4df4a89c8598aefcfd88
I20260505 07:54:20.749619 2268 log.cc:1199] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/e82e9ae417784b7e83f4843b129d0580
I20260505 07:54:20.747473 782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e0fcf1b5d8944a0b3ee0fe4e85f8608" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
W20260505 07:54:20.748734 718 leader_election.cc:343] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: b0d966bcd7f1419cbd090afc185ae78a
I20260505 07:54:20.749970 718 leader_election.cc:304] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:20.748517 1181 tablet_bootstrap.cc:492] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.748620 991 leader_election.cc:304] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.750396 1181 ts_tablet_manager.cc:1403] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.064s user 0.006s sys 0.000s
I20260505 07:54:20.750389 3363 raft_consensus.cc:2749] T b0d966bcd7f1419cbd090afc185ae78a P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.750711 3203 raft_consensus.cc:2804] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:20.750810 3203 raft_consensus.cc:493] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:20.750883 3203 raft_consensus.cc:3060] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.750582 745 tablet_service.cc:1558] Processing DeleteTablet for tablet 39eeecbfd10e4df4a89c8598aefcfd88 with delete_type TABLET_DATA_DELETED (replaced by tablet 9bd04d1dfca2456e8dde2f9dbc3c2082) from {username='slave'} at 127.0.0.1:55776
I20260505 07:54:20.751150 1024 tablet_service.cc:1558] Processing DeleteTablet for tablet 39eeecbfd10e4df4a89c8598aefcfd88 with delete_type TABLET_DATA_DELETED (replaced by tablet 9bd04d1dfca2456e8dde2f9dbc3c2082) from {username='slave'} at 127.0.0.1:47472
I20260505 07:54:20.751310 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 256) took 9774 ms. Trace:
I20260505 07:54:20.751435 885 tablet_service.cc:1558] Processing DeleteTablet for tablet 39eeecbfd10e4df4a89c8598aefcfd88 with delete_type TABLET_DATA_DELETED (replaced by tablet 9bd04d1dfca2456e8dde2f9dbc3c2082) from {username='slave'} at 127.0.0.1:53998
W20260505 07:54:20.751683 854 leader_election.cc:343] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.751785 854 leader_election.cc:304] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.751724 2447 rpcz_store.cc:276] 0505 07:54:10.976509 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.976558 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:20.650615 (+9674057us) ts_tablet_manager.cc:1151] Deleting tablet 02e06d8815034d91986cef12964860ea
0505 07:54:20.650625 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.678798 (+ 28173us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.681713 (+ 2915us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.751294 (+ 69581us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9673902}
I20260505 07:54:20.752064 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 254) took 9775 ms. Trace:
I20260505 07:54:20.752137 3296 raft_consensus.cc:2749] T 6e0fcf1b5d8944a0b3ee0fe4e85f8608 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.752135 2447 rpcz_store.cc:276] 0505 07:54:10.976323 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.976663 (+ 340us) service_pool.cc:224] Handling call
0505 07:54:20.751962 (+9775299us) ts_tablet_manager.cc:1151] Deleting tablet 909306e1e5f34b7c8fbc0d86cafce531
0505 07:54:20.751970 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.752020 (+ 50us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9775194}
I20260505 07:54:20.752367 2447 tablet_replica.cc:333] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.752556 2447 raft_consensus.cc:2243] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.752820 2447 raft_consensus.cc:2272] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.753085 1181 raft_consensus.cc:359] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.753312 1181 raft_consensus.cc:385] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.753404 1181 raft_consensus.cc:740] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.753351 3203 raft_consensus.cc:515] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.753598 3173 consensus_queue.cc:1048] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.753648 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 02e06d8815034d91986cef12964860ea (table default.Table_ca13a11f3fd249eabadc7213e7318acd [id=76602762055f42c0ad9805377eeb5d69]) successfully deleted
I20260505 07:54:20.753964 3203 leader_election.cc:290] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:20.753998 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 909306e1e5f34b7c8fbc0d86cafce531 because the tablet was not found. No further retry: Not found: Tablet not found: 909306e1e5f34b7c8fbc0d86cafce531
I20260505 07:54:20.754313 1181 consensus_queue.cc:260] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.754446 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84faf2e3d91e4cc394a3411c5b6c0bc7" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:20.754549 3296 raft_consensus.cc:493] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:20.754743 2447 ts_tablet_manager.cc:1916] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.754658 3296 raft_consensus.cc:515] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:20.754833 988 leader_election.cc:343] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.755065 1181 ts_tablet_manager.cc:1434] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260505 07:54:20.755250 3296 leader_election.cc:290] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.755347 1181 tablet_bootstrap.cc:492] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.755832 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.755982 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.756012 775 raft_consensus.cc:2468] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 2.
I20260505 07:54:20.756182 1047 raft_consensus.cc:2393] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d2799f5c6b4f4336b976ca9bbb458533 in current term 2: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:20.756453 854 leader_election.cc:304] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters:
I20260505 07:54:20.756937 1181 tablet_bootstrap.cc:654] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.757025 3296 raft_consensus.cc:2804] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260505 07:54:20.757129 3296 raft_consensus.cc:493] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Starting leader election (detected failure of leader eac7e1e8caba43cdb4df80cbb80e891c)
I20260505 07:54:20.757222 3296 raft_consensus.cc:3060] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Advancing to term 2
I20260505 07:54:20.760020 3296 raft_consensus.cc:515] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.757153 2268 ts_tablet_manager.cc:1950] T e82e9ae417784b7e83f4843b129d0580 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:20.760733 3296 leader_election.cc:290] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.760869 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:20.761039 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 404) took 3632 ms. Trace:
I20260505 07:54:20.761106 2268 rpcz_store.cc:276] 0505 07:54:17.128310 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.128347 (+ 37us) service_pool.cc:224] Handling call
0505 07:54:20.695327 (+3566980us) ts_tablet_manager.cc:1151] Deleting tablet e82e9ae417784b7e83f4843b129d0580
0505 07:54:20.695335 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.728869 (+ 33534us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.747409 (+ 18540us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.761022 (+ 13613us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3566850}
I20260505 07:54:20.761237 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f207a3ffbc94b01abd9e9d6837f5c81" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77"
I20260505 07:54:20.761418 1047 raft_consensus.cc:2393] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d63a3e535f8e4dd4a88e447253405b77 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate d2799f5c6b4f4336b976ca9bbb458533 in current term 2: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
I20260505 07:54:20.761457 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet e82e9ae417784b7e83f4843b129d0580 (table default.Table_9f8b49c83f7843318fd7a4645786e1d4 [id=a12c6d5dd696413490da09bf773c9a2f]) successfully deleted
I20260505 07:54:20.761555 2268 tablet_replica.cc:333] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:20.761735 2268 raft_consensus.cc:2243] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.761837 2268 raft_consensus.cc:2272] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.762380 2268 ts_tablet_manager.cc:1916] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.762401 3203 raft_consensus.cc:493] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.762499 3203 raft_consensus.cc:515] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.763044 3203 leader_election.cc:290] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:20.763160 775 raft_consensus.cc:2468] T 6f207a3ffbc94b01abd9e9d6837f5c81 P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2799f5c6b4f4336b976ca9bbb458533 in term 2.
I20260505 07:54:20.763285 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f4854e8752493fb832ae2c12a033ce" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.763419 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f4854e8752493fb832ae2c12a033ce" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.763554 918 raft_consensus.cc:2468] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
W20260505 07:54:20.763772 988 leader_election.cc:343] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.763856 854 leader_election.cc:304] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c; no voters: d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:20.763953 991 leader_election.cc:304] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.764204 3296 raft_consensus.cc:2804] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 FOLLOWER]: Leader election won for term 2
I20260505 07:54:20.764530 3296 raft_consensus.cc:697] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [term 2 LEADER]: Becoming Leader. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Running, Role: LEADER
I20260505 07:54:20.764787 3296 consensus_queue.cc:237] T 6f207a3ffbc94b01abd9e9d6837f5c81 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.765156 3203 raft_consensus.cc:2804] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:20.765262 3203 raft_consensus.cc:493] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:20.765362 3203 raft_consensus.cc:3060] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.768082 2447 ts_tablet_manager.cc:1929] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.768335 3203 raft_consensus.cc:515] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.769471 918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f4854e8752493fb832ae2c12a033ce" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:20.769667 918 raft_consensus.cc:3060] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.769361 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "24f4854e8752493fb832ae2c12a033ce" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:20.770637 2447 log.cc:1199] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/e82e9ae417784b7e83f4843b129d0580
I20260505 07:54:20.770963 2447 ts_tablet_manager.cc:1950] T e82e9ae417784b7e83f4843b129d0580 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
W20260505 07:54:20.771108 988 leader_election.cc:343] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.771320 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "84faf2e3d91e4cc394a3411c5b6c0bc7" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:20.771472 907 raft_consensus.cc:3060] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.771726 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 255) took 9795 ms. Trace:
I20260505 07:54:20.772485 918 raft_consensus.cc:2468] T 24f4854e8752493fb832ae2c12a033ce P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:20.772989 991 leader_election.cc:304] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.773022 885 tablet_service.cc:1511] Processing CreateTablet for tablet 4e3c3a1a357942508ef0e6b48bfa6523 (DEFAULT_TABLE table=default.Table_aeab2a6ff10d466f8126362debcb6db6 [id=433968907e944cb58739e748649261d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.773286 2447 rpcz_store.cc:276] 0505 07:54:10.976407 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.976735 (+ 328us) service_pool.cc:224] Handling call
0505 07:54:20.752329 (+9775594us) ts_tablet_manager.cc:1151] Deleting tablet e82e9ae417784b7e83f4843b129d0580
0505 07:54:20.752337 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.759456 (+ 7119us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.768017 (+ 8561us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.771692 (+ 3675us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9775470}
I20260505 07:54:20.773460 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e3c3a1a357942508ef0e6b48bfa6523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.773491 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 4e3c3a1a357942508ef0e6b48bfa6523 (DEFAULT_TABLE table=default.Table_aeab2a6ff10d466f8126362debcb6db6 [id=433968907e944cb58739e748649261d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.773792 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e3c3a1a357942508ef0e6b48bfa6523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.773911 2447 tablet_replica.cc:333] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.774088 2447 raft_consensus.cc:2243] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.774226 2447 raft_consensus.cc:2272] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.775846 3173 raft_consensus.cc:2804] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:20.775967 3173 raft_consensus.cc:697] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:20.776199 3173 consensus_queue.cc:237] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.777302 2447 ts_tablet_manager.cc:1916] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.777428 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet e82e9ae417784b7e83f4843b129d0580 (table default.Table_9f8b49c83f7843318fd7a4645786e1d4 [id=a12c6d5dd696413490da09bf773c9a2f]) successfully deleted
W20260505 07:54:20.778730 718 consensus_peers.cc:597] T e5eaa91ecc1f43d18b313b6be4cc211e P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5eaa91ecc1f43d18b313b6be4cc211e. This is attempt 1: this message will repeat every 5th retry.
W20260505 07:54:20.779081 3464 log.cc:927] Time spent T 172ca66c49354f549fa55fe63abf9970 P d63a3e535f8e4dd4a88e447253405b77: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260505 07:54:20.780344 3203 leader_election.cc:290] T 24f4854e8752493fb832ae2c12a033ce P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839)
I20260505 07:54:20.780859 3361 raft_consensus.cc:493] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.780979 3361 raft_consensus.cc:515] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.781451 3361 leader_election.cc:290] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:20.782105 2276 ts_tablet_manager.cc:1929] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.782189 2276 log.cc:1199] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/9f541eff399641dd81a0771186805493
I20260505 07:54:20.782438 2276 ts_tablet_manager.cc:1950] T 9f541eff399641dd81a0771186805493 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:20.783104 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 259) took 6233 ms. Trace:
I20260505 07:54:20.783190 2276 rpcz_store.cc:276] 0505 07:54:14.549506 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.549558 (+ 52us) service_pool.cc:224] Handling call
0505 07:54:20.681816 (+6132258us) ts_tablet_manager.cc:1151] Deleting tablet 9f541eff399641dd81a0771186805493
0505 07:54:20.681841 (+ 25us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.723317 (+ 41476us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.782065 (+ 58748us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.783090 (+ 1025us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6132094}
I20260505 07:54:20.783674 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf77f23a2ec647faa715fb912db3a995" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
I20260505 07:54:20.783984 1322 catalog_manager.cc:5002] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): tablet 9f541eff399641dd81a0771186805493 (table default.Table_a86abb495238445381ac0202804b4ca0 [id=4d3b183ab74a4078beb7583c9e60e2c6]) successfully deleted
W20260505 07:54:20.784250 856 leader_election.cc:343] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
W20260505 07:54:20.784698 991 consensus_peers.cc:597] T b0d966bcd7f1419cbd090afc185ae78a P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b0d966bcd7f1419cbd090afc185ae78a. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:20.788745 2447 ts_tablet_manager.cc:1929] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.789254 3202 raft_consensus.cc:493] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.789390 3202 raft_consensus.cc:515] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.789922 3202 leader_election.cc:290] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:20.789408 1355 catalog_manager.cc:6741] ReplaceTablet: tablet 39eeecbfd10e4df4a89c8598aefcfd88 deleted and replaced by tablet 9bd04d1dfca2456e8dde2f9dbc3c2082
I20260505 07:54:20.790454 2447 log.cc:1199] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/2d86b8b4dadf4fd2b6409a8cb70e313e
I20260505 07:54:20.790510 1355 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:46452 (request call id 279) took 2103 ms. Trace:
I20260505 07:54:20.790647 745 tablet_service.cc:1511] Processing CreateTablet for tablet 4e3c3a1a357942508ef0e6b48bfa6523 (DEFAULT_TABLE table=default.Table_aeab2a6ff10d466f8126362debcb6db6 [id=433968907e944cb58739e748649261d8]), partition=RANGE (key) PARTITION UNBOUNDED
W20260505 07:54:20.790885 991 leader_election.cc:343] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: b8a85b65b1904bd6b793af48f71ed726
I20260505 07:54:20.791013 772 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a85b65b1904bd6b793af48f71ed726" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.791177 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e3c3a1a357942508ef0e6b48bfa6523. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.791636 2447 ts_tablet_manager.cc:1950] T 2d86b8b4dadf4fd2b6409a8cb70e313e P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.790843 1355 rpcz_store.cc:276] 0505 07:54:18.686959 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:20.383396 (+1696437us) service_pool.cc:224] Handling call
0505 07:54:20.790466 (+407070us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:20.439457 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:20.439561 (+ 104us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:20.439566 (+ 5us) write_op.cc:435] Acquired schema lock
0505 07:54:20.439568 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:20.439686 (+ 118us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:20.439698 (+ 12us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:20.439702 (+ 4us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:20.439797 (+ 95us) tablet.cc:701] Row locks acquired
0505 07:54:20.439800 (+ 3us) write_op.cc:260] PREPARE: finished
0505 07:54:20.439832 (+ 32us) write_op.cc:270] Start()
0505 07:54:20.439861 (+ 29us) write_op.cc:276] Timestamp: P: 1777967660439763 usec, L: 0
0505 07:54:20.439864 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:20.440019 (+ 155us) log.cc:844] Serialized 622 byte log entry
0505 07:54:20.443203 (+ 3184us) op_driver.cc:464] REPLICATION: finished
0505 07:54:20.443327 (+ 124us) write_op.cc:301] APPLY: starting
0505 07:54:20.443341 (+ 14us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:20.443379 (+ 38us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:20.443403 (+ 24us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:20.443496 (+ 93us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:20.443558 (+ 62us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0505 07:54:20.443565 (+ 7us) write_op.cc:312] APPLY: finished
0505 07:54:20.443656 (+ 91us) log.cc:844] Serialized 29 byte log entry
0505 07:54:20.443714 (+ 58us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.443732 (+ 18us) write_op.cc:454] Released schema lock
0505 07:54:20.443738 (+ 6us) write_op.cc:341] FINISH: Updating metrics
0505 07:54:20.443817 (+ 79us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.443821 (+ 4us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":81,"apply.run_cpu_time_us":481,"apply.run_wall_time_us":480,"num_ops":2,"prepare.queue_time_us":46,"prepare.run_cpu_time_us":659,"prepare.run_wall_time_us":659,"raft.queue_time_us":718,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":242,"replication_time_us":3313}]]}
I20260505 07:54:20.793064 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 257) took 9816 ms. Trace:
I20260505 07:54:20.793044 1024 tablet_service.cc:1511] Processing CreateTablet for tablet cd55b9f5917e4198b7c104fd6324cbcf (DEFAULT_TABLE table=default.Table_aa752db71710464497d37acff57357ef [id=130e36e8a2b4411086137fe8ef07c2b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.793345 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd55b9f5917e4198b7c104fd6324cbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.793877 2447 rpcz_store.cc:276] 0505 07:54:10.976617 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.976879 (+ 262us) service_pool.cc:224] Handling call
0505 07:54:20.773866 (+9796987us) ts_tablet_manager.cc:1151] Deleting tablet 2d86b8b4dadf4fd2b6409a8cb70e313e
0505 07:54:20.773877 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.780782 (+ 6905us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.788704 (+ 7922us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.793048 (+ 4344us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9796848}
I20260505 07:54:20.794064 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf77f23a2ec647faa715fb912db3a995" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.794603 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 258) took 9817 ms. Trace:
W20260505 07:54:20.794864 854 leader_election.cc:343] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.794983 854 leader_election.cc:304] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.794957 2447 rpcz_store.cc:276] 0505 07:54:10.977256 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.977347 (+ 91us) service_pool.cc:224] Handling call
0505 07:54:20.794530 (+9817183us) ts_tablet_manager.cc:1151] Deleting tablet aab3850799254dda9716fd5bf9404159
0505 07:54:20.794538 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.794590 (+ 52us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9817055}
I20260505 07:54:20.795405 3361 raft_consensus.cc:2749] T cf77f23a2ec647faa715fb912db3a995 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.795475 2447 tablet_replica.cc:333] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.795686 2447 raft_consensus.cc:2243] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.794497 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 2d86b8b4dadf4fd2b6409a8cb70e313e (table default.Table_ff8ae2444af4400eb86af8cc0fa3c146 [id=6a242917efbf4ae5a5fa82539d6f7b34]) successfully deleted
W20260505 07:54:20.796377 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet aab3850799254dda9716fd5bf9404159 because the tablet was not found. No further retry: Not found: Tablet not found: aab3850799254dda9716fd5bf9404159
W20260505 07:54:20.796540 988 leader_election.cc:343] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.796622 988 leader_election.cc:304] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.796244 2447 raft_consensus.cc:2272] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.797075 3202 raft_consensus.cc:2749] T b8a85b65b1904bd6b793af48f71ed726 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.798084 2447 ts_tablet_manager.cc:1916] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.798692 885 tablet_service.cc:1511] Processing CreateTablet for tablet cd55b9f5917e4198b7c104fd6324cbcf (DEFAULT_TABLE table=default.Table_aa752db71710464497d37acff57357ef [id=130e36e8a2b4411086137fe8ef07c2b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.799222 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd55b9f5917e4198b7c104fd6324cbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.806042 1181 tablet_bootstrap.cc:492] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.806248 1181 ts_tablet_manager.cc:1403] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.051s user 0.005s sys 0.000s
I20260505 07:54:20.807214 1181 raft_consensus.cc:359] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.807359 2447 ts_tablet_manager.cc:1929] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.807438 2447 log.cc:1199] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/abe5f6f0774545b296bf1b03a08e145d
I20260505 07:54:20.807511 1181 raft_consensus.cc:385] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.807646 1181 raft_consensus.cc:740] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.807895 1355 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46452:
name: "default.Table_827def018b924bebb4837283ef3997b3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260505 07:54:20.808028 1181 consensus_queue.cc:260] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.808790 1181 ts_tablet_manager.cc:1434] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:20.809119 1181 tablet_bootstrap.cc:492] T ae829daad64545e687a9528e5c38b9ed P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.809249 745 tablet_service.cc:1511] Processing CreateTablet for tablet cd55b9f5917e4198b7c104fd6324cbcf (DEFAULT_TABLE table=default.Table_aa752db71710464497d37acff57357ef [id=130e36e8a2b4411086137fe8ef07c2b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.810820 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd55b9f5917e4198b7c104fd6324cbcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.811273 2447 ts_tablet_manager.cc:1950] T abe5f6f0774545b296bf1b03a08e145d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.811787 2268 ts_tablet_manager.cc:1929] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260505 07:54:20.812103 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 259) took 9834 ms. Trace:
I20260505 07:54:20.812223 2268 log.cc:1199] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/ad5c570d4861460d8ab0c8ff8a7a3a03
I20260505 07:54:20.813371 2447 rpcz_store.cc:276] 0505 07:54:10.977370 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:10.977496 (+ 126us) service_pool.cc:224] Handling call
0505 07:54:20.795437 (+9817941us) ts_tablet_manager.cc:1151] Deleting tablet abe5f6f0774545b296bf1b03a08e145d
0505 07:54:20.795445 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.801002 (+ 5557us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.807320 (+ 6318us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.812084 (+ 4764us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9817821}
I20260505 07:54:20.813491 2268 ts_tablet_manager.cc:1950] T ad5c570d4861460d8ab0c8ff8a7a3a03 P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
W20260505 07:54:20.814018 1355 catalog_manager.cc:7033] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_827def018b924bebb4837283ef3997b3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260505 07:54:20.814132 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 406) took 3629 ms. Trace:
I20260505 07:54:20.815459 2268 rpcz_store.cc:276] 0505 07:54:17.184373 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.184420 (+ 47us) service_pool.cc:224] Handling call
0505 07:54:20.761516 (+3577096us) ts_tablet_manager.cc:1151] Deleting tablet ad5c570d4861460d8ab0c8ff8a7a3a03
0505 07:54:20.761525 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.791442 (+ 29917us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.811744 (+ 20302us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.814114 (+ 2370us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3576908}
I20260505 07:54:20.816210 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 1d174694d5c14b7aa5d93fd4ea181692 (DEFAULT_TABLE table=default.Table_a2dcfcc5bec246db8d5426a08af47ed9 [id=2c40e72fd2d04a9d967ba74d69801959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.816284 3202 consensus_queue.cc:1048] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.815629 772 raft_consensus.cc:3060] T 6ea1d3829ede4b5582a022b64265df7e P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.816076 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet ad5c570d4861460d8ab0c8ff8a7a3a03 (table default.Table_d17cd5686eb94c3b8993d4310f0d31f7 [id=8c0e89be595c4eb2b4d4103121cdccf0]) successfully deleted
I20260505 07:54:20.817251 892 tablet_service.cc:1511] Processing CreateTablet for tablet 1d174694d5c14b7aa5d93fd4ea181692 (DEFAULT_TABLE table=default.Table_a2dcfcc5bec246db8d5426a08af47ed9 [id=2c40e72fd2d04a9d967ba74d69801959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.817579 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d174694d5c14b7aa5d93fd4ea181692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.817889 907 raft_consensus.cc:2468] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:20.819384 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet abe5f6f0774545b296bf1b03a08e145d (table default.Table_c22dca26148c48f882eeea850f50bea4 [id=6c12928213864a1195f14e35d819f339]) successfully deleted
I20260505 07:54:20.819501 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 260) took 9820 ms. Trace:
I20260505 07:54:20.819551 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d174694d5c14b7aa5d93fd4ea181692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.819579 2447 rpcz_store.cc:276] 0505 07:54:10.999427 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.002538 (+ 3111us) service_pool.cc:224] Handling call
0505 07:54:20.819392 (+9816854us) ts_tablet_manager.cc:1151] Deleting tablet 880e326cbea74233a0ec5abb31f5b0d7
0505 07:54:20.819403 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.819487 (+ 84us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9816679}
W20260505 07:54:20.820391 1315 catalog_manager.cc:4982] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 880e326cbea74233a0ec5abb31f5b0d7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 880e326cbea74233a0ec5abb31f5b0d7 already in progress: creating tablet
I20260505 07:54:20.820444 2447 tablet_replica.cc:333] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.820626 2447 raft_consensus.cc:2243] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.821511 2447 raft_consensus.cc:2272] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.822414 991 leader_election.cc:304] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.822410 754 tablet_service.cc:1511] Processing CreateTablet for tablet 1d174694d5c14b7aa5d93fd4ea181692 (DEFAULT_TABLE table=default.Table_a2dcfcc5bec246db8d5426a08af47ed9 [id=2c40e72fd2d04a9d967ba74d69801959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.822680 2447 ts_tablet_manager.cc:1916] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.822726 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d174694d5c14b7aa5d93fd4ea181692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.822858 3202 raft_consensus.cc:2804] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:20.822959 3202 raft_consensus.cc:697] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:20.823160 3202 consensus_queue.cc:237] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.828557 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 7914da4a063448a9ad2d9b87cf95513a (DEFAULT_TABLE table=default.Table_e754ed46e4014d0ba9652ccb3db35817 [id=1604d786796044c7af074b85654f8b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.828927 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7914da4a063448a9ad2d9b87cf95513a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.835305 2447 ts_tablet_manager.cc:1929] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.836120 2447 log.cc:1199] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/472441bd29704be59aabb0f7a4964bb0
I20260505 07:54:20.836624 2447 ts_tablet_manager.cc:1950] T 472441bd29704be59aabb0f7a4964bb0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.836773 3203 consensus_queue.cc:1048] T 6ea1d3829ede4b5582a022b64265df7e P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.839557 892 tablet_service.cc:1511] Processing CreateTablet for tablet 7914da4a063448a9ad2d9b87cf95513a (DEFAULT_TABLE table=default.Table_e754ed46e4014d0ba9652ccb3db35817 [id=1604d786796044c7af074b85654f8b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.840013 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7914da4a063448a9ad2d9b87cf95513a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.812983 1181 tablet_bootstrap.cc:654] T ae829daad64545e687a9528e5c38b9ed P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.845233 3361 consensus_queue.cc:1048] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.846412 1355 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:46452 (request call id 280) took 2142 ms. Trace:
I20260505 07:54:20.846796 2268 tablet_replica.cc:333] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:20.847031 2268 raft_consensus.cc:2243] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.847270 2268 pending_rounds.cc:70] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: Trying to abort 1 pending ops.
I20260505 07:54:20.847345 2268 pending_rounds.cc:77] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7282555536263680000 op_type: NO_OP noop_request { }
I20260505 07:54:20.847461 2268 raft_consensus.cc:2889] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:20.847568 2268 raft_consensus.cc:2272] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.847862 754 tablet_service.cc:1511] Processing CreateTablet for tablet 7914da4a063448a9ad2d9b87cf95513a (DEFAULT_TABLE table=default.Table_e754ed46e4014d0ba9652ccb3db35817 [id=1604d786796044c7af074b85654f8b9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.848286 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7914da4a063448a9ad2d9b87cf95513a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.848498 2268 ts_tablet_manager.cc:1916] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.851703 1028 tablet_service.cc:1511] Processing CreateTablet for tablet a4692bc7a7064643b0595b47c906e2e2 (DEFAULT_TABLE table=default.Table_79228e2b3f8b49b68041c5310cc30d88 [id=c8c2456bb6ff44c890b8db2e8bf012eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.852190 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4692bc7a7064643b0595b47c906e2e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.854492 896 tablet_service.cc:1511] Processing CreateTablet for tablet a4692bc7a7064643b0595b47c906e2e2 (DEFAULT_TABLE table=default.Table_79228e2b3f8b49b68041c5310cc30d88 [id=c8c2456bb6ff44c890b8db2e8bf012eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.854900 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4692bc7a7064643b0595b47c906e2e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.858201 3203 consensus_queue.cc:1048] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.862620 1176 tablet_bootstrap.cc:492] T f2f5b72f58a24db0b58c15b47227bc60 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:20.862749 1176 ts_tablet_manager.cc:1403] T f2f5b72f58a24db0b58c15b47227bc60 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.127s user 0.003s sys 0.000s
I20260505 07:54:20.863510 1176 raft_consensus.cc:359] T f2f5b72f58a24db0b58c15b47227bc60 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
W20260505 07:54:20.867058 718 consensus_peers.cc:597] T c4dd3ece5cb34ceb82feeb67e75da0b5 P eac7e1e8caba43cdb4df80cbb80e891c -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4dd3ece5cb34ceb82feeb67e75da0b5. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:20.870043 775 raft_consensus.cc:3060] T 4a8e6019406845ebb29051432a96dfd1 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.870616 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 261) took 9863 ms. Trace:
I20260505 07:54:20.871258 1181 tablet_bootstrap.cc:492] T ae829daad64545e687a9528e5c38b9ed P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.872215 2447 rpcz_store.cc:276] 0505 07:54:11.006885 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.007025 (+ 140us) service_pool.cc:224] Handling call
0505 07:54:20.820389 (+9813364us) ts_tablet_manager.cc:1151] Deleting tablet 472441bd29704be59aabb0f7a4964bb0
0505 07:54:20.820398 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.826355 (+ 5957us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.835264 (+ 8909us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.870599 (+ 35335us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9813164}
I20260505 07:54:20.872867 1181 ts_tablet_manager.cc:1403] T ae829daad64545e687a9528e5c38b9ed P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.064s user 0.005s sys 0.001s
I20260505 07:54:20.873262 2447 tablet_replica.cc:333] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.873355 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 472441bd29704be59aabb0f7a4964bb0 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]) successfully deleted
I20260505 07:54:20.873771 3144 consensus_queue.cc:1048] T 4a8e6019406845ebb29051432a96dfd1 P d2799f5c6b4f4336b976ca9bbb458533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.874241 2447 raft_consensus.cc:2243] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.848990 1355 rpcz_store.cc:276] 0505 07:54:18.703470 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:20.797676 (+2094206us) service_pool.cc:224] Handling call
0505 07:54:20.817472 (+ 19796us) catalog_manager.cc:2101] Acquired catalog manager lock
0505 07:54:20.822492 (+ 5020us) catalog_manager.cc:2147] Created new table and tablet info
0505 07:54:20.846097 (+ 23605us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.846102 (+ 5us) write_op.cc:454] Released schema lock
0505 07:54:20.846176 (+ 74us) catalog_manager.cc:2207] Wrote table and tablets to system table
0505 07:54:20.846284 (+ 108us) catalog_manager.cc:2241] Inserted table and tablets into CatalogManager maps
0505 07:54:20.846370 (+ 86us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0505 07:54:20.834605 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0505 07:54:20.834728 (+ 123us) write_op.cc:432] Acquiring schema lock in shared mode
0505 07:54:20.834735 (+ 7us) write_op.cc:435] Acquired schema lock
0505 07:54:20.834737 (+ 2us) tablet.cc:662] Decoding operations
0505 07:54:20.834859 (+ 122us) write_op.cc:620] Acquiring the partition lock for write op
0505 07:54:20.834873 (+ 14us) write_op.cc:641] Partition lock acquired for write op
0505 07:54:20.834878 (+ 5us) tablet.cc:685] Acquiring locks for 2 operations
0505 07:54:20.834963 (+ 85us) tablet.cc:701] Row locks acquired
0505 07:54:20.834965 (+ 2us) write_op.cc:260] PREPARE: finished
0505 07:54:20.834997 (+ 32us) write_op.cc:270] Start()
0505 07:54:20.835032 (+ 35us) write_op.cc:276] Timestamp: P: 1777967660834943 usec, L: 0
0505 07:54:20.835035 (+ 3us) op_driver.cc:348] REPLICATION: starting
0505 07:54:20.835228 (+ 193us) log.cc:844] Serialized 433 byte log entry
0505 07:54:20.838990 (+ 3762us) op_driver.cc:464] REPLICATION: finished
0505 07:54:20.839152 (+ 162us) write_op.cc:301] APPLY: starting
0505 07:54:20.839172 (+ 20us) tablet.cc:1367] starting BulkCheckPresence
0505 07:54:20.839225 (+ 53us) tablet.cc:1370] finished BulkCheckPresence
0505 07:54:20.839259 (+ 34us) tablet.cc:1372] starting ApplyRowOperation cycle
0505 07:54:20.839420 (+ 161us) tablet.cc:1383] finished ApplyRowOperation cycle
0505 07:54:20.839479 (+ 59us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0505 07:54:20.839491 (+ 12us) write_op.cc:312] APPLY: finished
0505 07:54:20.839633 (+ 142us) log.cc:844] Serialized 29 byte log entry
0505 07:54:20.839710 (+ 77us) write_op.cc:489] Releasing partition, row and schema locks
0505 07:54:20.839735 (+ 25us) write_op.cc:454] Released schema lock
0505 07:54:20.839743 (+ 8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":103,"apply.run_cpu_time_us":662,"apply.run_wall_time_us":660,"num_ops":2,"prepare.queue_time_us":11513,"prepare.run_cpu_time_us":744,"prepare.run_wall_time_us":744,"raft.queue_time_us":1011,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":246,"replication_time_us":3918}]]}
I20260505 07:54:20.876019 2447 raft_consensus.cc:2272] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.877660 1355 master_service.cc:954] ReplaceTablet: received request to replace tablet 3a73dbb979344805876efbafbf1b2985 from {username='slave'} at 127.0.0.1:46452
I20260505 07:54:20.876788 1181 raft_consensus.cc:359] T ae829daad64545e687a9528e5c38b9ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.878384 1181 raft_consensus.cc:385] T ae829daad64545e687a9528e5c38b9ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.878625 1181 raft_consensus.cc:740] T ae829daad64545e687a9528e5c38b9ed P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.879101 1181 consensus_queue.cc:260] T ae829daad64545e687a9528e5c38b9ed P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.879336 2447 ts_tablet_manager.cc:1916] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.880538 744 tablet_service.cc:1511] Processing CreateTablet for tablet a4692bc7a7064643b0595b47c906e2e2 (DEFAULT_TABLE table=default.Table_79228e2b3f8b49b68041c5310cc30d88 [id=c8c2456bb6ff44c890b8db2e8bf012eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.880972 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4692bc7a7064643b0595b47c906e2e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.881337 1181 ts_tablet_manager.cc:1434] T ae829daad64545e687a9528e5c38b9ed P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20260505 07:54:20.884418 896 tablet_service.cc:1511] Processing CreateTablet for tablet e71f220bcbc143119d4471b2ec146817 (DEFAULT_TABLE table=default.Table_969875795fc0470298d7aecc64918154 [id=7e4e731011c2436f926dac0ab5453216]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.881493 1024 tablet_service.cc:1511] Processing CreateTablet for tablet e71f220bcbc143119d4471b2ec146817 (DEFAULT_TABLE table=default.Table_969875795fc0470298d7aecc64918154 [id=7e4e731011c2436f926dac0ab5453216]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.884791 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71f220bcbc143119d4471b2ec146817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.884873 1181 tablet_bootstrap.cc:492] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.887156 1181 tablet_bootstrap.cc:654] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.887722 3364 consensus_queue.cc:1048] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.890028 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71f220bcbc143119d4471b2ec146817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.896131 2447 ts_tablet_manager.cc:1929] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.890281 2268 ts_tablet_manager.cc:1929] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.897434 2268 log.cc:1199] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/21b48208e68247e78c12f0fec7ba07dc
I20260505 07:54:20.899467 2268 ts_tablet_manager.cc:1950] T 21b48208e68247e78c12f0fec7ba07dc P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:20.901124 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 407) took 3716 ms. Trace:
I20260505 07:54:20.902913 2268 rpcz_store.cc:276] 0505 07:54:17.184802 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.184848 (+ 46us) service_pool.cc:224] Handling call
0505 07:54:20.846747 (+3661899us) ts_tablet_manager.cc:1151] Deleting tablet 21b48208e68247e78c12f0fec7ba07dc
0505 07:54:20.846759 (+ 12us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.851256 (+ 4497us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.890232 (+ 38976us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.901103 (+ 10871us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3630918}
I20260505 07:54:20.905735 1172 tablet_bootstrap.cc:492] T 0d86424d75b4491d8967ee0c0203b5ca P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.906354 1172 ts_tablet_manager.cc:1403] T 0d86424d75b4491d8967ee0c0203b5ca P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.202s user 0.017s sys 0.039s
I20260505 07:54:20.906816 2268 tablet_replica.cc:333] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:20.907014 2268 raft_consensus.cc:2243] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.907162 2268 raft_consensus.cc:2272] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.783530 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 260) took 6206 ms. Trace:
I20260505 07:54:20.907248 2276 rpcz_store.cc:276] 0505 07:54:14.577332 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.577389 (+ 57us) service_pool.cc:224] Handling call
0505 07:54:20.783453 (+6206064us) ts_tablet_manager.cc:1151] Deleting tablet 4258df74ea0841449846cdf1f6e00b9b
0505 07:54:20.783462 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.783517 (+ 55us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6205897}
I20260505 07:54:20.907768 2268 ts_tablet_manager.cc:1916] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.908843 1172 raft_consensus.cc:359] T 0d86424d75b4491d8967ee0c0203b5ca P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.909080 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 21b48208e68247e78c12f0fec7ba07dc (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]) successfully deleted
I20260505 07:54:20.909893 3202 raft_consensus.cc:493] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.910048 3202 raft_consensus.cc:515] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.910724 3202 leader_election.cc:290] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:20.912586 1172 raft_consensus.cc:385] T 0d86424d75b4491d8967ee0c0203b5ca P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.912827 2447 log.cc:1199] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/f842d8ac6b2142ffa69945c6087be34b
I20260505 07:54:20.913487 1172 raft_consensus.cc:740] T 0d86424d75b4491d8967ee0c0203b5ca P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.914021 2447 ts_tablet_manager.cc:1950] T f842d8ac6b2142ffa69945c6087be34b P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.914682 1172 consensus_queue.cc:260] T 0d86424d75b4491d8967ee0c0203b5ca P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.915908 1181 tablet_bootstrap.cc:492] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:20.915752 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b12d2d2b866348d781209fa787da7d7e" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.917227 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 262) took 9874 ms. Trace:
I20260505 07:54:20.917322 2447 rpcz_store.cc:276] 0505 07:54:11.043012 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.043061 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:20.873220 (+9830159us) ts_tablet_manager.cc:1151] Deleting tablet f842d8ac6b2142ffa69945c6087be34b
0505 07:54:20.873230 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.884092 (+ 10862us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.896082 (+ 11990us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.917206 (+ 21124us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9829963}
I20260505 07:54:20.917721 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet f842d8ac6b2142ffa69945c6087be34b (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]) successfully deleted
W20260505 07:54:20.918210 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 4258df74ea0841449846cdf1f6e00b9b because the tablet was not found. No further retry: Not found: Tablet not found: 4258df74ea0841449846cdf1f6e00b9b
I20260505 07:54:20.918751 1181 ts_tablet_manager.cc:1403] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.003s
I20260505 07:54:20.919282 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 261) took 6341 ms. Trace:
I20260505 07:54:20.919392 2276 rpcz_store.cc:276] 0505 07:54:14.577485 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.577591 (+ 106us) service_pool.cc:224] Handling call
0505 07:54:20.917888 (+6340297us) ts_tablet_manager.cc:1151] Deleting tablet 7001482d90364dbdb8b9c5bfb98e1cd4
0505 07:54:20.919193 (+ 1305us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.919266 (+ 73us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6340171}
I20260505 07:54:20.919540 1181 raft_consensus.cc:359] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.919736 1181 raft_consensus.cc:385] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.919801 1181 raft_consensus.cc:740] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.918102 1172 ts_tablet_manager.cc:1434] T 0d86424d75b4491d8967ee0c0203b5ca P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.011s user 0.001s sys 0.000s
I20260505 07:54:20.920806 1172 tablet_bootstrap.cc:492] T 35cd0e809d6345a6a9825d4dc44d7575 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:20.922869 1172 tablet_bootstrap.cc:654] T 35cd0e809d6345a6a9825d4dc44d7575 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
W20260505 07:54:20.923552 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 7001482d90364dbdb8b9c5bfb98e1cd4 because the tablet was not found. No further retry: Not found: Tablet not found: 7001482d90364dbdb8b9c5bfb98e1cd4
I20260505 07:54:20.924057 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 262) took 6341 ms. Trace:
I20260505 07:54:20.924182 2276 rpcz_store.cc:276] 0505 07:54:14.582214 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.582268 (+ 54us) service_pool.cc:224] Handling call
0505 07:54:20.923943 (+6341675us) ts_tablet_manager.cc:1151] Deleting tablet e29ba55a846548679cb3b86305d0d6ed
0505 07:54:20.923956 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.924036 (+ 80us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6341479}
I20260505 07:54:20.924495 2447 tablet_replica.cc:333] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.924780 2447 raft_consensus.cc:2243] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 2 LEADER]: Raft consensus shutting down.
I20260505 07:54:20.925132 2447 pending_rounds.cc:70] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: Trying to abort 1 pending ops.
I20260505 07:54:20.925189 2447 pending_rounds.cc:77] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7282555529933807616 op_type: NO_OP noop_request { }
W20260505 07:54:20.925226 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet e29ba55a846548679cb3b86305d0d6ed because the tablet was not found. No further retry: Not found: Tablet not found: e29ba55a846548679cb3b86305d0d6ed
I20260505 07:54:20.925287 2447 raft_consensus.cc:2889] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260505 07:54:20.925376 2447 raft_consensus.cc:2272] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.926930 1181 consensus_queue.cc:260] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:20.927801 2268 ts_tablet_manager.cc:1929] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:20.927909 2268 log.cc:1199] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/5191f9a357de4607aab6c7a8b4bcccdb
I20260505 07:54:20.928377 2268 ts_tablet_manager.cc:1950] T 5191f9a357de4607aab6c7a8b4bcccdb P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:20.928650 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 263) took 6346 ms. Trace:
I20260505 07:54:20.928689 1181 ts_tablet_manager.cc:1434] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260505 07:54:20.928980 1181 tablet_bootstrap.cc:492] T aacf715cb07d4fe9b2c57caea594ab87 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:20.929112 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 411) took 3720 ms. Trace:
I20260505 07:54:20.929201 2268 rpcz_store.cc:276] 0505 07:54:17.208326 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.208362 (+ 36us) service_pool.cc:224] Handling call
0505 07:54:20.906771 (+3698409us) ts_tablet_manager.cc:1151] Deleting tablet 5191f9a357de4607aab6c7a8b4bcccdb
0505 07:54:20.906782 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.922240 (+ 15458us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.927752 (+ 5512us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.929088 (+ 1336us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3698257}
I20260505 07:54:20.929553 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 410) took 3721 ms. Trace:
I20260505 07:54:20.929613 2268 rpcz_store.cc:276] 0505 07:54:17.208233 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.208290 (+ 57us) service_pool.cc:224] Handling call
0505 07:54:20.929445 (+3721155us) ts_tablet_manager.cc:1151] Deleting tablet 66f069982138460da3b7ae7a0b4e3833
0505 07:54:20.929454 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.929539 (+ 85us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3720831}
I20260505 07:54:20.929906 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 409) took 3721 ms. Trace:
I20260505 07:54:20.929971 2268 rpcz_store.cc:276] 0505 07:54:17.208118 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:17.209519 (+ 1401us) service_pool.cc:224] Handling call
0505 07:54:20.929813 (+3720294us) ts_tablet_manager.cc:1151] Deleting tablet 212e496d564a4b5ebabb22504be76bf2
0505 07:54:20.929821 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.929894 (+ 73us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3720043}
I20260505 07:54:20.930246 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 412) took 2917 ms. Trace:
I20260505 07:54:20.930302 2268 rpcz_store.cc:276] 0505 07:54:18.013280 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.013333 (+ 53us) service_pool.cc:224] Handling call
0505 07:54:20.930155 (+2916822us) ts_tablet_manager.cc:1151] Deleting tablet 152176572c904b8296073afad0585817
0505 07:54:20.930163 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.930231 (+ 68us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2916667}
I20260505 07:54:20.930567 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 415) took 2916 ms. Trace:
I20260505 07:54:20.930624 2268 rpcz_store.cc:276] 0505 07:54:18.014134 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.014168 (+ 34us) service_pool.cc:224] Handling call
0505 07:54:20.930475 (+2916307us) ts_tablet_manager.cc:1151] Deleting tablet 8cf3c334ebb24956b50dbefc23d4d60b
0505 07:54:20.930484 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.930556 (+ 72us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2916185}
I20260505 07:54:20.930898 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 416) took 2916 ms. Trace:
I20260505 07:54:20.930982 1181 tablet_bootstrap.cc:654] T aacf715cb07d4fe9b2c57caea594ab87 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.925994 2447 ts_tablet_manager.cc:1916] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260505 07:54:20.933737 988 leader_election.cc:343] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.930955 2268 rpcz_store.cc:276] 0505 07:54:18.014436 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.014475 (+ 39us) service_pool.cc:224] Handling call
0505 07:54:20.930812 (+2916337us) ts_tablet_manager.cc:1151] Deleting tablet 9c4fc18f6faa488882d9bffb30793cdd
0505 07:54:20.930820 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.930883 (+ 63us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2916216}
I20260505 07:54:20.936775 2268 tablet_replica.cc:333] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:20.936954 2268 raft_consensus.cc:2243] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.937188 2268 raft_consensus.cc:2272] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.937301 3366 consensus_queue.cc:1048] T 09019fdf60374b11bf3427f21a55b5e0 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.937652 2268 ts_tablet_manager.cc:1916] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.939618 2276 rpcz_store.cc:276] 0505 07:54:14.582351 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.582480 (+ 129us) service_pool.cc:224] Handling call
0505 07:54:20.928559 (+6346079us) ts_tablet_manager.cc:1151] Deleting tablet 36b33d265ba04e93993c7e0a4efea1a4
0505 07:54:20.928570 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.928634 (+ 64us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6345954}
W20260505 07:54:20.940040 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 36b33d265ba04e93993c7e0a4efea1a4 because the tablet was not found. No further retry: Not found: Tablet not found: 36b33d265ba04e93993c7e0a4efea1a4
I20260505 07:54:20.941813 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 264) took 6359 ms. Trace:
I20260505 07:54:20.943064 2276 rpcz_store.cc:276] 0505 07:54:14.582499 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:14.582620 (+ 121us) service_pool.cc:224] Handling call
0505 07:54:20.941743 (+6359123us) ts_tablet_manager.cc:1151] Deleting tablet 02412333633a4cdebe95a144fa6c363c
0505 07:54:20.941751 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.941801 (+ 50us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6359019}
I20260505 07:54:20.943442 3490 raft_consensus.cc:493] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:20.943617 3490 raft_consensus.cc:515] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
W20260505 07:54:20.944013 1322 catalog_manager.cc:4969] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): delete failed for tablet 02412333633a4cdebe95a144fa6c363c because the tablet was not found. No further retry: Not found: Tablet not found: 02412333633a4cdebe95a144fa6c363c
I20260505 07:54:20.944304 3490 leader_election.cc:290] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:20.944569 2276 tablet_replica.cc:333] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533: stopping tablet replica
I20260505 07:54:20.944903 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af8af6d1f5d9429083002991e3470f7b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:20.945528 2276 raft_consensus.cc:2243] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260505 07:54:20.946389 988 leader_election.cc:343] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:20.863730 1176 raft_consensus.cc:385] T f2f5b72f58a24db0b58c15b47227bc60 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:20.946617 1176 raft_consensus.cc:740] T f2f5b72f58a24db0b58c15b47227bc60 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:20.946892 754 tablet_service.cc:1511] Processing CreateTablet for tablet e71f220bcbc143119d4471b2ec146817 (DEFAULT_TABLE table=default.Table_969875795fc0470298d7aecc64918154 [id=7e4e731011c2436f926dac0ab5453216]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.947763 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 5191f9a357de4607aab6c7a8b4bcccdb (table default.Table_94e6f1ee4ab947d8bc9864ab0c8bd524 [id=f7b2051b32e240c1856be3f0802b2f3e]) successfully deleted
W20260505 07:54:20.948026 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 66f069982138460da3b7ae7a0b4e3833 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66f069982138460da3b7ae7a0b4e3833 already in progress: creating tablet
W20260505 07:54:20.948254 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 212e496d564a4b5ebabb22504be76bf2 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 212e496d564a4b5ebabb22504be76bf2 already in progress: creating tablet
W20260505 07:54:20.948453 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 152176572c904b8296073afad0585817 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 152176572c904b8296073afad0585817 already in progress: creating tablet
W20260505 07:54:20.948630 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 8cf3c334ebb24956b50dbefc23d4d60b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cf3c334ebb24956b50dbefc23d4d60b already in progress: creating tablet
W20260505 07:54:20.948866 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 9c4fc18f6faa488882d9bffb30793cdd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9c4fc18f6faa488882d9bffb30793cdd already in progress: creating tablet
W20260505 07:54:20.949548 991 leader_election.cc:343] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Not found: Tablet not found: b12d2d2b866348d781209fa787da7d7e
I20260505 07:54:20.949694 991 leader_election.cc:304] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d63a3e535f8e4dd4a88e447253405b77; no voters: d2799f5c6b4f4336b976ca9bbb458533, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.950139 3490 raft_consensus.cc:2749] T b12d2d2b866348d781209fa787da7d7e P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:20.950366 3490 consensus_queue.cc:1048] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:20.949096 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af8af6d1f5d9429083002991e3470f7b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:20.953238 907 raft_consensus.cc:2468] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:20.953608 991 leader_election.cc:304] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:20.950430 2268 ts_tablet_manager.cc:1929] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:20.953994 3490 raft_consensus.cc:2804] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:20.954228 3490 raft_consensus.cc:493] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:20.954329 3490 raft_consensus.cc:3060] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:20.954159 2268 log.cc:1199] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/83e62b7af9b847b7a9005e4345905c0d
I20260505 07:54:20.955214 2268 ts_tablet_manager.cc:1950] T 83e62b7af9b847b7a9005e4345905c0d P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:20.950119 1176 consensus_queue.cc:260] T f2f5b72f58a24db0b58c15b47227bc60 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:20.957175 2276 raft_consensus.cc:2272] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.957337 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 417) took 2942 ms. Trace:
I20260505 07:54:20.957412 2268 rpcz_store.cc:276] 0505 07:54:18.014754 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.014789 (+ 35us) service_pool.cc:224] Handling call
0505 07:54:20.936738 (+2921949us) ts_tablet_manager.cc:1151] Deleting tablet 83e62b7af9b847b7a9005e4345905c0d
0505 07:54:20.936746 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.943060 (+ 6314us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.950392 (+ 7332us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.957322 (+ 6930us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2921818}
I20260505 07:54:20.958058 2276 ts_tablet_manager.cc:1916] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.959780 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 83e62b7af9b847b7a9005e4345905c0d (table default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]) successfully deleted
I20260505 07:54:20.960146 1028 tablet_service.cc:1511] Processing CreateTablet for tablet 4d91e1ea0af54dd2abf83785e1cb00ba (DEFAULT_TABLE table=default.Table_145774eb3d744a92b4b09f7a8ee032f5 [id=04d376fa36d94370ba8fc60623700695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.960649 1028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d91e1ea0af54dd2abf83785e1cb00ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.963855 896 tablet_service.cc:1511] Processing CreateTablet for tablet a47814e51cbb45128ade31488e75bf5d (DEFAULT_TABLE table=default.Table_c42ce47a2e564486bb790f681399d8e3 [id=fb62e9cab2514f8ca65ed39c134c342f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.964313 896 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a47814e51cbb45128ade31488e75bf5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.968106 885 tablet_service.cc:1511] Processing CreateTablet for tablet 4d91e1ea0af54dd2abf83785e1cb00ba (DEFAULT_TABLE table=default.Table_145774eb3d744a92b4b09f7a8ee032f5 [id=04d376fa36d94370ba8fc60623700695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.968202 1024 tablet_service.cc:1511] Processing CreateTablet for tablet a47814e51cbb45128ade31488e75bf5d (DEFAULT_TABLE table=default.Table_c42ce47a2e564486bb790f681399d8e3 [id=fb62e9cab2514f8ca65ed39c134c342f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.968500 885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d91e1ea0af54dd2abf83785e1cb00ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.968505 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a47814e51cbb45128ade31488e75bf5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.956502 754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e71f220bcbc143119d4471b2ec146817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.971812 1018 tablet_service.cc:1511] Processing CreateTablet for tablet ff781e3c9abe41ea940020b4febd47ce (DEFAULT_TABLE table=default.Table_12419a358167451da817d86794464e07 [id=8b62c703b3bf4fdf896128455e786228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.972152 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff781e3c9abe41ea940020b4febd47ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.976462 971 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:20.961537 744 tablet_service.cc:1511] Processing CreateTablet for tablet a47814e51cbb45128ade31488e75bf5d (DEFAULT_TABLE table=default.Table_c42ce47a2e564486bb790f681399d8e3 [id=fb62e9cab2514f8ca65ed39c134c342f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.956609 1176 ts_tablet_manager.cc:1434] T f2f5b72f58a24db0b58c15b47227bc60 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.094s user 0.002s sys 0.000s
I20260505 07:54:20.978148 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a47814e51cbb45128ade31488e75bf5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.978566 1176 tablet_bootstrap.cc:492] T 24f4854e8752493fb832ae2c12a033ce P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:20.981022 1176 tablet_bootstrap.cc:654] T 24f4854e8752493fb832ae2c12a033ce P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:20.981523 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 420) took 2949 ms. Trace:
I20260505 07:54:20.981663 1172 tablet_bootstrap.cc:492] T 35cd0e809d6345a6a9825d4dc44d7575 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:20.981598 2268 rpcz_store.cc:276] 0505 07:54:18.032126 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.032169 (+ 43us) service_pool.cc:224] Handling call
0505 07:54:20.957653 (+2925484us) ts_tablet_manager.cc:1151] Deleting tablet 99850198f4d04607b52953342daa753e
0505 07:54:20.957664 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.981506 (+ 23842us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2925316}
I20260505 07:54:20.981854 2268 tablet_replica.cc:333] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:20.982030 2268 raft_consensus.cc:2243] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.982167 2268 raft_consensus.cc:2272] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.982599 2268 ts_tablet_manager.cc:1916] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.982825 750 tablet_service.cc:1511] Processing CreateTablet for tablet ff781e3c9abe41ea940020b4febd47ce (DEFAULT_TABLE table=default.Table_12419a358167451da817d86794464e07 [id=8b62c703b3bf4fdf896128455e786228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.983098 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff781e3c9abe41ea940020b4febd47ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.984947 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9cbfb2345a1a4b218c2dddbae6667e47 (table default.Table_92a11daebfd54782b9dbf6af7af4f214 [id=34b0c1456db3424da7730ffd60760ebb]). Expected version 1 got 0
I20260505 07:54:20.959390 745 tablet_service.cc:1511] Processing CreateTablet for tablet 4d91e1ea0af54dd2abf83785e1cb00ba (DEFAULT_TABLE table=default.Table_145774eb3d744a92b4b09f7a8ee032f5 [id=04d376fa36d94370ba8fc60623700695]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.969743 2447 ts_tablet_manager.cc:1929] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260505 07:54:20.990065 2447 log.cc:1199] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/df1b62a30b2842b897a04c793bb5e7ce
I20260505 07:54:20.990250 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d91e1ea0af54dd2abf83785e1cb00ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.990423 2447 ts_tablet_manager.cc:1950] T df1b62a30b2842b897a04c793bb5e7ce P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:20.991153 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 263) took 9939 ms. Trace:
I20260505 07:54:20.991235 2447 rpcz_store.cc:276] 0505 07:54:11.052071 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.052128 (+ 57us) service_pool.cc:224] Handling call
0505 07:54:20.924438 (+9872310us) ts_tablet_manager.cc:1151] Deleting tablet df1b62a30b2842b897a04c793bb5e7ce
0505 07:54:20.924451 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.942441 (+ 17990us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.969693 (+ 27252us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.991135 (+ 21442us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9872122}
I20260505 07:54:20.991544 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 264) took 9936 ms. Trace:
I20260505 07:54:20.991628 2447 rpcz_store.cc:276] 0505 07:54:11.055610 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.055702 (+ 92us) service_pool.cc:224] Handling call
0505 07:54:20.991469 (+9935767us) ts_tablet_manager.cc:1151] Deleting tablet 297537a92684425d8508f7b3b4c9d379
0505 07:54:20.991477 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.991531 (+ 54us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9935607}
I20260505 07:54:20.991865 2447 tablet_replica.cc:333] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:20.992043 2447 raft_consensus.cc:2243] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:20.992161 2447 raft_consensus.cc:2272] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:20.992581 2447 ts_tablet_manager.cc:1916] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:20.995980 892 tablet_service.cc:1511] Processing CreateTablet for tablet ff781e3c9abe41ea940020b4febd47ce (DEFAULT_TABLE table=default.Table_12419a358167451da817d86794464e07 [id=8b62c703b3bf4fdf896128455e786228]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.996968 899 tablet_service.cc:1511] Processing CreateTablet for tablet 7ec59a5261704d4c862f1ae3c84d8dde (DEFAULT_TABLE table=default.Table_2b5ac39d20eb4deeb452b514f976ba97 [id=bf4514184f6d4b819703f8d31505b52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:20.997129 892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff781e3c9abe41ea940020b4febd47ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:20.997308 899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ec59a5261704d4c862f1ae3c84d8dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:21.000027 750 tablet_service.cc:1511] Processing CreateTablet for tablet 7ec59a5261704d4c862f1ae3c84d8dde (DEFAULT_TABLE table=default.Table_2b5ac39d20eb4deeb452b514f976ba97 [id=bf4514184f6d4b819703f8d31505b52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:21.000427 750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ec59a5261704d4c862f1ae3c84d8dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:21.002991 2447 ts_tablet_manager.cc:1929] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:21.003093 2447 log.cc:1199] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/080b2984fcf2478a8def6540da63554d
I20260505 07:54:21.003603 2447 ts_tablet_manager.cc:1950] T 080b2984fcf2478a8def6540da63554d P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:21.003496 3366 consensus_queue.cc:1048] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:21.004348 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 266) took 9948 ms. Trace:
I20260505 07:54:21.004443 2447 rpcz_store.cc:276] 0505 07:54:11.055859 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.055903 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:20.991832 (+9935929us) ts_tablet_manager.cc:1151] Deleting tablet 080b2984fcf2478a8def6540da63554d
0505 07:54:20.991840 (+ 8us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.995314 (+ 3474us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.002946 (+ 7632us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.004330 (+ 1384us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9935799}
I20260505 07:54:21.004964 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 265) took 9949 ms. Trace:
I20260505 07:54:21.005057 2447 rpcz_store.cc:276] 0505 07:54:11.055727 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.055893 (+ 166us) service_pool.cc:224] Handling call
0505 07:54:21.004855 (+9948962us) ts_tablet_manager.cc:1151] Deleting tablet 6ca2852cee84476dbe5e5072a43087a1
0505 07:54:21.004866 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:21.004946 (+ 80us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9948566}
I20260505 07:54:21.005525 2447 tablet_replica.cc:333] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:21.005846 2447 raft_consensus.cc:2243] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:21.006037 2447 raft_consensus.cc:2272] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:21.000496 2268 ts_tablet_manager.cc:1929] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:21.008210 2268 log.cc:1199] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/29a82ed2b87c487aac84194f2dcb0d6b
I20260505 07:54:20.999794 3490 raft_consensus.cc:515] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.008561 2268 ts_tablet_manager.cc:1950] T 29a82ed2b87c487aac84194f2dcb0d6b P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:21.009678 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 422) took 2977 ms. Trace:
I20260505 07:54:21.009797 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af8af6d1f5d9429083002991e3470f7b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:21.009758 2268 rpcz_store.cc:276] 0505 07:54:18.032662 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.032713 (+ 51us) service_pool.cc:224] Handling call
0505 07:54:20.981811 (+2949098us) ts_tablet_manager.cc:1151] Deleting tablet 29a82ed2b87c487aac84194f2dcb0d6b
0505 07:54:20.981822 (+ 11us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:20.989550 (+ 7728us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:20.999692 (+ 10142us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.009661 (+ 9969us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2948958}
I20260505 07:54:21.009999 1176 tablet_bootstrap.cc:492] T 24f4854e8752493fb832ae2c12a033ce P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:21.010111 1176 ts_tablet_manager.cc:1403] T 24f4854e8752493fb832ae2c12a033ce P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.032s user 0.003s sys 0.000s
W20260505 07:54:21.010321 988 leader_election.cc:343] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:21.010596 1102 heartbeater.cc:499] Master 127.0.105.61:33565 was elected leader, sending a full tablet report...
I20260505 07:54:21.010783 1176 raft_consensus.cc:359] T 24f4854e8752493fb832ae2c12a033ce P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:21.010936 1176 raft_consensus.cc:385] T 24f4854e8752493fb832ae2c12a033ce P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:21.010988 1176 raft_consensus.cc:740] T 24f4854e8752493fb832ae2c12a033ce P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:21.012639 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af8af6d1f5d9429083002991e3470f7b" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:21.012791 907 raft_consensus.cc:3060] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:21.013200 2268 tablet_replica.cc:333] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:21.013386 2268 raft_consensus.cc:2243] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:21.013458 3489 raft_consensus.cc:493] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:21.013545 3489 raft_consensus.cc:515] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.013677 2268 raft_consensus.cc:2272] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:21.014058 3489 leader_election.cc:290] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:21.014161 2268 ts_tablet_manager.cc:1916] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:21.014585 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180157193a184add848ba67ca8042109" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:21.014662 1176 consensus_queue.cc:260] T 24f4854e8752493fb832ae2c12a033ce P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:21.014745 912 raft_consensus.cc:2468] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:21.014959 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180157193a184add848ba67ca8042109" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:21.015107 991 leader_election.cc:304] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:21.015318 3489 raft_consensus.cc:2804] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:21.015376 3489 raft_consensus.cc:493] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:21.015452 3490 leader_election.cc:290] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:21.015408 988 leader_election.cc:343] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:21.015427 3489 raft_consensus.cc:3060] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:21.017696 2447 ts_tablet_manager.cc:1916] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:21.018559 3489 raft_consensus.cc:515] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.019320 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180157193a184add848ba67ca8042109" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:21.019471 912 raft_consensus.cc:3060] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:21.020550 1181 tablet_bootstrap.cc:492] T aacf715cb07d4fe9b2c57caea594ab87 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:21.020673 1181 ts_tablet_manager.cc:1403] T aacf715cb07d4fe9b2c57caea594ab87 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.092s user 0.004s sys 0.016s
I20260505 07:54:21.021061 2268 ts_tablet_manager.cc:1929] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260505 07:54:21.021196 2268 log.cc:1199] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/dd3fc8a9d9f04636b1f31bb88411c1da
I20260505 07:54:21.021441 1181 raft_consensus.cc:359] T aacf715cb07d4fe9b2c57caea594ab87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.021602 1181 raft_consensus.cc:385] T aacf715cb07d4fe9b2c57caea594ab87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:21.021661 1181 raft_consensus.cc:740] T aacf715cb07d4fe9b2c57caea594ab87 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:21.022375 3366 consensus_queue.cc:1048] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:21.022600 2268 ts_tablet_manager.cc:1950] T dd3fc8a9d9f04636b1f31bb88411c1da P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:21.024778 3004 consensus_queue.cc:1048] T 59a3cd77808c4e858ea2ef9ed2a28c38 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:21.021881 1181 consensus_queue.cc:260] T aacf715cb07d4fe9b2c57caea594ab87 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.025352 1181 ts_tablet_manager.cc:1434] T aacf715cb07d4fe9b2c57caea594ab87 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260505 07:54:21.029444 887 tablet_service.cc:1511] Processing CreateTablet for tablet 66d19dc2258b45468ae4b6ed8b22407e (DEFAULT_TABLE table=default.Table_26167dd500974094b3adfce3f4247587 [id=f4fa96cfb8894d108cb124ef1c96206e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:21.029866 887 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d19dc2258b45468ae4b6ed8b22407e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:21.031949 1176 ts_tablet_manager.cc:1434] T 24f4854e8752493fb832ae2c12a033ce P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.022s user 0.002s sys 0.000s
I20260505 07:54:21.033015 894 tablet_service.cc:1511] Processing CreateTablet for tablet e9292a6b344a431da3e4ca203b82931e (DEFAULT_TABLE table=default.Table_a63821ea5fc747bdb435a2f668326eea [id=11b8e4c8970a4a069dfd72a07ea8071f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:21.033339 894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9292a6b344a431da3e4ca203b82931e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260505 07:54:20.995376 1318 catalog_manager.cc:4982] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): delete failed for tablet 99850198f4d04607b52953342daa753e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 99850198f4d04607b52953342daa753e already in progress: creating tablet
I20260505 07:54:21.034282 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet 29a82ed2b87c487aac84194f2dcb0d6b (table default.Table_7b4336be586a4fc1a8dada3651641fd9 [id=057b807a46cb43b4a60dc61212a8b8a8]) successfully deleted
I20260505 07:54:21.001528 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet df1b62a30b2842b897a04c793bb5e7ce (table default.Table_6bd554849539457bb38dbbef32deb609 [id=269f2d2935a644518501cb0700a7cff9]) successfully deleted
W20260505 07:54:21.034600 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 297537a92684425d8508f7b3b4c9d379 because the tablet was not found. No further retry: Not found: Tablet not found: 297537a92684425d8508f7b3b4c9d379
I20260505 07:54:21.034793 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet 080b2984fcf2478a8def6540da63554d (table default.Table_575ab5b128ee4a1baebbe1f90baf005a [id=cdb574c7a4e14a5ea7f3c8b036a8c336]) successfully deleted
W20260505 07:54:21.034976 1315 catalog_manager.cc:4969] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): delete failed for tablet 6ca2852cee84476dbe5e5072a43087a1 because the tablet was not found. No further retry: Not found: Tablet not found: 6ca2852cee84476dbe5e5072a43087a1
I20260505 07:54:21.035713 1172 ts_tablet_manager.cc:1403] T 35cd0e809d6345a6a9825d4dc44d7575 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.115s user 0.002s sys 0.004s
I20260505 07:54:21.036473 1172 raft_consensus.cc:359] T 35cd0e809d6345a6a9825d4dc44d7575 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:21.036623 1172 raft_consensus.cc:385] T 35cd0e809d6345a6a9825d4dc44d7575 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:21.036684 1172 raft_consensus.cc:740] T 35cd0e809d6345a6a9825d4dc44d7575 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:21.036903 1172 consensus_queue.cc:260] T 35cd0e809d6345a6a9825d4dc44d7575 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:21.037159 744 tablet_service.cc:1511] Processing CreateTablet for tablet e9292a6b344a431da3e4ca203b82931e (DEFAULT_TABLE table=default.Table_a63821ea5fc747bdb435a2f668326eea [id=11b8e4c8970a4a069dfd72a07ea8071f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:21.037398 1172 ts_tablet_manager.cc:1434] T 35cd0e809d6345a6a9825d4dc44d7575 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260505 07:54:21.037480 744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9292a6b344a431da3e4ca203b82931e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:21.037703 1172 tablet_bootstrap.cc:492] T 152176572c904b8296073afad0585817 P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:21.039834 1018 tablet_service.cc:1511] Processing CreateTablet for tablet 7ec59a5261704d4c862f1ae3c84d8dde (DEFAULT_TABLE table=default.Table_2b5ac39d20eb4deeb452b514f976ba97 [id=bf4514184f6d4b819703f8d31505b52a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:21.040230 1024 tablet_service.cc:1511] Processing CreateTablet for tablet 66d19dc2258b45468ae4b6ed8b22407e (DEFAULT_TABLE table=default.Table_26167dd500974094b3adfce3f4247587 [id=f4fa96cfb8894d108cb124ef1c96206e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:21.041056 1032 tablet_service.cc:1511] Processing CreateTablet for tablet e9292a6b344a431da3e4ca203b82931e (DEFAULT_TABLE table=default.Table_a63821ea5fc747bdb435a2f668326eea [id=11b8e4c8970a4a069dfd72a07ea8071f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:21.042521 907 raft_consensus.cc:2468] T af8af6d1f5d9429083002991e3470f7b P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:21.043157 1172 tablet_bootstrap.cc:654] T 152176572c904b8296073afad0585817 P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:21.044159 1176 tablet_bootstrap.cc:492] T fba1d19f902a4a0a8b581135f6611129 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:21.044889 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet abadd3af175e466db81491b8a69a6d7d (table default.Table_ab0d444d71bd4422a0c2ab8e6340fc5c [id=1dc1ab0d9aa44adba092a585e5b98640]). Expected version 1 got 0
I20260505 07:54:21.045475 912 raft_consensus.cc:2468] T 180157193a184add848ba67ca8042109 P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:21.046590 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 18e909ee93a24ea9842e95357c2ba858 (table default.Table_61b1474d21b44e19918a653005f920a1 [id=2a512c2695b540c2a9e14616ee257125]). Expected version 1 got 0
I20260505 07:54:21.047098 991 leader_election.cc:304] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:21.047343 3173 raft_consensus.cc:2804] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:21.047449 3173 raft_consensus.cc:697] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:21.047691 3173 consensus_queue.cc:237] T af8af6d1f5d9429083002991e3470f7b P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.048090 1176 tablet_bootstrap.cc:654] T fba1d19f902a4a0a8b581135f6611129 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:21.049844 3483 consensus_queue.cc:1048] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:21.052055 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e243711164f04faabbde53b0a09d90b1 (table default.Table_a3cc6e739a71414f9f925aa759a07bd3 [id=5993b31e5350440f9715f1e416b90eb2]). Expected version 1 got 0
I20260505 07:54:21.052407 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet b6607b7428044e428af9d765c9c9b3ba (table default.Table_d3f58b97177c4772a1b934bd1b4c3856 [id=35a56d340665450ab03ba49b845ae78a]). Expected version 1 got 0
I20260505 07:54:21.056061 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 03260a8507804e548486c81225de3ad9 (table default.Table_115e3886a51546409c5e744efd8b0462 [id=3c21c289d8f94c70b4ba7ef64dabc072]). Expected version 1 got 0
I20260505 07:54:21.058967 3199 consensus_queue.cc:1048] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:21.059556 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d5083f98e8e14ccf99a19cdf2bc59018 (table default.Table_a2ff5add35c641e9b5341948a1b297c8 [id=b1689bb0b68c4d21b1689c04b2904022]). Expected version 3 got 0
I20260505 07:54:21.060387 3483 raft_consensus.cc:493] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:21.060532 3483 raft_consensus.cc:515] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.060599 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet bdfadf8810f842698d96b6c13ffa1216 (table default.Table_6432df5b02b140d89f970c85cc4f1b91 [id=dcf4d78fda334683bf51e07167c9999a]). Expected version 1 got 0
I20260505 07:54:21.060873 1018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ec59a5261704d4c862f1ae3c84d8dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:21.061040 3483 leader_election.cc:290] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:21.063719 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 64ca343bd32a4eeb80901bc3157b3b6e (table default.Table_ba009e4831a44471ac8a72fffba6674a [id=ce5a935268d844f4878670e02bff2dd3]). Expected version 1 got 0
I20260505 07:54:21.063835 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676e2109e59749b58931d597ce1beb9c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:21.064239 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 24f4854e8752493fb832ae2c12a033ce (table default.Table_018af517d4374faaa03e4eba33184d90 [id=1a1c2d94983e409c8b75400de8adbae8]). Expected version 1 got 0
W20260505 07:54:21.064610 988 leader_election.cc:343] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:21.065007 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 7235a3ccef904913bbd67c403aac3a87 (table default.Table_55c9c867105547fc9295e1dc46d9a865 [id=5bb0990470284a3f8c9af2108d5bbeb4]). Expected version 1 got 0
I20260505 07:54:21.065573 2447 ts_tablet_manager.cc:1929] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:21.065696 2447 log.cc:1199] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/f200abef13ac42698a9b96fc8af82ff0
I20260505 07:54:21.065893 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a1175f7deec14fe39098ec89ab3bc049 (table default.Table_14ab8ad3d70847b3a8fbf3bec625e1fd [id=bccdbf7ecfdd40c59dc52c4562f88728]). Expected version 2 got 0
I20260505 07:54:21.066076 2447 ts_tablet_manager.cc:1950] T f200abef13ac42698a9b96fc8af82ff0 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:21.066881 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 267) took 10009 ms. Trace:
I20260505 07:54:21.067062 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 5006965919824adbbb23d744fb9ff335 (table default.Table_acc94c9e49fb47cf96fa3a0bcf267ccd [id=6dad63e723444d899e014cf136aeff89]). Expected version 1 got 0
I20260505 07:54:21.066973 2447 rpcz_store.cc:276] 0505 07:54:11.057655 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.057699 (+ 44us) service_pool.cc:224] Handling call
0505 07:54:21.005465 (+9947766us) ts_tablet_manager.cc:1151] Deleting tablet f200abef13ac42698a9b96fc8af82ff0
0505 07:54:21.005479 (+ 14us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:21.025689 (+ 20210us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.065522 (+ 39833us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.066862 (+ 1340us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":9947621}
I20260505 07:54:21.067323 2447 tablet_replica.cc:333] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:21.067536 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet f200abef13ac42698a9b96fc8af82ff0 (table default.Table_157815480b75424885451ec833e3438d [id=ebd83e4e101d4282ab8aeec21c281c60]) successfully deleted
I20260505 07:54:21.069262 1349 catalog_manager.cc:5671] T f36a0f68fc2b4adda04ec3b9db5f2d8e P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1). New cstate: current_term: 1 leader_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:21.075158 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f36a0f68fc2b4adda04ec3b9db5f2d8e (table default.Table_4a8d555d38d449c6805a868165a8ab82 [id=d6192f5cec174951bf17af2e65d01f79]). Expected version 1 got 0
I20260505 07:54:21.076675 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1f043cd9042f4e1180d7c0cd90713b03 (table default.Table_e027e2467d834e1897181fba6338e7ef [id=606b29c8a5a4482cbb8bd1c86b3e7d8c]). Expected version 1 got 0
I20260505 07:54:21.067534 2447 raft_consensus.cc:2243] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:21.077508 1176 tablet_bootstrap.cc:492] T fba1d19f902a4a0a8b581135f6611129 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:21.077768 1176 ts_tablet_manager.cc:1403] T fba1d19f902a4a0a8b581135f6611129 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.000s
I20260505 07:54:21.078735 1176 raft_consensus.cc:359] T fba1d19f902a4a0a8b581135f6611129 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.078955 1176 raft_consensus.cc:385] T fba1d19f902a4a0a8b581135f6611129 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:21.079047 1176 raft_consensus.cc:740] T fba1d19f902a4a0a8b581135f6611129 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:21.079314 1176 consensus_queue.cc:260] T fba1d19f902a4a0a8b581135f6611129 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.080782 1176 ts_tablet_manager.cc:1434] T fba1d19f902a4a0a8b581135f6611129 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260505 07:54:21.081354 1176 tablet_bootstrap.cc:492] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:21.083804 1176 tablet_bootstrap.cc:654] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:21.084378 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0c4342c2c6d142ad90b0270608007eda (table default.Table_006d19a5a2b34032acd01663bdcc34e8 [id=658005bd9e754a2abedba00b3f7cf8ec]). Expected version 1 got 0
I20260505 07:54:21.084744 991 leader_election.cc:304] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:21.085165 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 310ef5aef7e44ec5a062a01d2a175a11 (table default.Table_823e01a26aa7450e869f0132a92ce378 [id=77518ca228af4a6698c5d908a0c76b50]). Expected version 1 got 0
I20260505 07:54:21.085237 3261 raft_consensus.cc:2804] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:21.085485 3261 raft_consensus.cc:697] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:21.085682 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 6e0fcf1b5d8944a0b3ee0fe4e85f8608 (table default.Table_23cfcb1d56bc4e72af90bc263a464bf2 [id=77cced330f69467cb7e54e309ff5a0db]). Expected version 1 got 0
I20260505 07:54:21.085796 3261 consensus_queue.cc:237] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.086160 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet f1ba93d114e746ef80697aa1401dd108 (table default.Table_001ba75f284249c3b0b479b5b7e27f4c [id=75f94c485caa4e4394a34f1399c9769d]). Expected version 2 got 0
I20260505 07:54:21.087563 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet de1a667b14fe494c8ee4be0f47a340be (table default.Table_53d2dcb1d87247a6add98d671bfa470b [id=037074511cd8426487cfdb5298d1ee8e]). Expected version 3 got 2
I20260505 07:54:21.087960 1349 catalog_manager.cc:5671] T 68e8f9f7b30f4912af916c15f9ae25d8 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:21.089920 1024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d19dc2258b45468ae4b6ed8b22407e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:21.092351 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676e2109e59749b58931d597ce1beb9c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:21.092526 912 raft_consensus.cc:2468] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:21.077533 2447 raft_consensus.cc:2272] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260505 07:54:21.093502 991 consensus_peers.cc:597] T d896e290167b4d0081e8e8b831c8ae2a P d63a3e535f8e4dd4a88e447253405b77 -> Peer d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839): Couldn't send request to peer d2799f5c6b4f4336b976ca9bbb458533. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d896e290167b4d0081e8e8b831c8ae2a. This is attempt 1: this message will repeat every 5th retry.
I20260505 07:54:21.093649 991 leader_election.cc:304] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:21.093894 2447 ts_tablet_manager.cc:1916] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:21.094234 3490 raft_consensus.cc:2804] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:21.094308 3490 raft_consensus.cc:493] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:21.094372 3490 raft_consensus.cc:3060] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:21.094460 1032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9292a6b344a431da3e4ca203b82931e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:21.089382 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e8f9f7b30f4912af916c15f9ae25d8 (table default.Table_b20b461c86aa4e52b4580432defb5b96 [id=5bf31c7aad3d40b0a73a40a14ca8c04d]). Expected version 2 got 0
I20260505 07:54:21.096665 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 4b9963a0f1a54ee9a73486bb1801ee74 (table default.Table_fc4b6502e4974457b4123c5ac53d567c [id=eddd2f477fd142228488291699aa5e4f]). Expected version 1 got 0
I20260505 07:54:21.096710 3490 raft_consensus.cc:515] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.096906 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1bd5951bfc3346c2a6aad8f33357394a (table default.Table_4e651b0d817f408e8e36b0399a57a04f [id=d85dae5f1f424012929e0d90ef8b2d22]). Expected version 1 got 0
I20260505 07:54:21.097151 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet fba1d19f902a4a0a8b581135f6611129 (table default.Table_db92d1d6790447508a5f3b466214a285 [id=c82e4e1fd6004306906c95957b86620f]). Expected version 1 got 0
I20260505 07:54:21.097625 1181 tablet_bootstrap.cc:492] T d86214559d014f028acc5a9691efc9ca P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:21.098121 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 38eee32a6fd84f89a1e88074f16d4242 (table default.Table_94a492cf2db14ff3900cf09b17058af3 [id=4231aa55ac4e481eb686252d30d91195]). Expected version 1 got 0
I20260505 07:54:21.098459 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 180157193a184add848ba67ca8042109 (table default.Table_3c38db9697fd4fafa0d5a58a07d4e244 [id=ba1b026067294e0cae324fb994815400]). Expected version 1 got 0
I20260505 07:54:21.099107 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 090fbe6c480249338e7210050770adeb (table default.Table_a2ac48b8b2a34b6580b27ea49735f549 [id=4f7e2ee9cb0f4c2fb2a46f7a472ddff9]). Expected version 1 got 0
I20260505 07:54:21.099169 1181 tablet_bootstrap.cc:654] T d86214559d014f028acc5a9691efc9ca P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:21.099905 3490 leader_election.cc:290] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
I20260505 07:54:21.100682 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676e2109e59749b58931d597ce1beb9c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
W20260505 07:54:21.103052 988 leader_election.cc:343] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:21.104041 3363 consensus_queue.cc:1048] T 6efb07cb60f64dfead739a3ca1343df9 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:21.104722 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 89d3e08134704859abc44257f484b8cd (table default.Table_e9f1ff58ca6d41f19d05c458dc529941 [id=7579207045ec4220b966af9016a8f050]). Expected version 1 got 0
I20260505 07:54:21.111063 1349 catalog_manager.cc:5671] T 225c56ac920b448c89c38d6fb3c1bdef P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3). New cstate: current_term: 1 leader_uuid: "d63a3e535f8e4dd4a88e447253405b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } }
I20260505 07:54:21.111599 1176 tablet_bootstrap.cc:492] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:21.111775 1176 ts_tablet_manager.cc:1403] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.030s user 0.002s sys 0.003s
I20260505 07:54:21.111770 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e1e1d42408fc4aa3a3c7e8ec67eb7dc6 (table default.Table_1899c1d796874753b600a2185d9e3088 [id=93dfee1a134944188b041e9281fa16c3]). Expected version 3 got 0
I20260505 07:54:21.112136 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 423) took 3079 ms. Trace:
I20260505 07:54:21.112229 2268 rpcz_store.cc:276] 0505 07:54:18.033049 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.033098 (+ 49us) service_pool.cc:224] Handling call
0505 07:54:21.009986 (+2976888us) ts_tablet_manager.cc:1151] Deleting tablet dd3fc8a9d9f04636b1f31bb88411c1da
0505 07:54:21.013164 (+ 3178us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:21.017591 (+ 4427us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.021020 (+ 3429us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.112116 (+ 91096us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1453,"tablet-delete.queue_time_us":2976727}
I20260505 07:54:21.112576 2268 tablet_replica.cc:333] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77: stopping tablet replica
I20260505 07:54:21.112551 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 260a8d41de464e429491ada4bfe2c761 (table default.Table_647fec9e0fb846cd86a882ea54f6a79a [id=199b3d61d922405b9e2a018c1c8c6080]). Expected version 1 got 0
I20260505 07:54:21.112502 1176 raft_consensus.cc:359] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.112695 1176 raft_consensus.cc:385] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:21.112761 1176 raft_consensus.cc:740] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:21.112783 2268 raft_consensus.cc:2243] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Raft consensus shutting down.
I20260505 07:54:21.113003 1176 consensus_queue.cc:260] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.113469 2268 raft_consensus.cc:2272] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:21.113929 1176 ts_tablet_manager.cc:1434] T 84faf2e3d91e4cc394a3411c5b6c0bc7 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260505 07:54:21.113591 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 90612f0afb4949f491e625c61e40d247 (table default.Table_61a6fc82edd5408f991f40a8d15394a5 [id=9930805fdf364f25aea11f859b50bc71]). Expected version 1 got 0
I20260505 07:54:21.114326 1176 tablet_bootstrap.cc:492] T b8a85b65b1904bd6b793af48f71ed726 P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:21.114722 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676e2109e59749b58931d597ce1beb9c" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533"
I20260505 07:54:21.114920 912 raft_consensus.cc:3060] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:21.116546 1176 tablet_bootstrap.cc:654] T b8a85b65b1904bd6b793af48f71ed726 P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:21.116698 2447 ts_tablet_manager.cc:1929] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:21.116849 2447 log.cc:1199] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-0/wal/wals/d7e0c8ad716e443588d4ae3aa28ed4bf
I20260505 07:54:21.117249 2447 ts_tablet_manager.cc:1950] T d7e0c8ad716e443588d4ae3aa28ed4bf P eac7e1e8caba43cdb4df80cbb80e891c: Deleting consensus metadata
I20260505 07:54:21.117873 912 raft_consensus.cc:2468] T 676e2109e59749b58931d597ce1beb9c P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 1.
I20260505 07:54:21.118170 2447 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55776 (request call id 269) took 10060 ms. Trace:
I20260505 07:54:21.118280 2447 rpcz_store.cc:276] 0505 07:54:11.057917 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:11.057953 (+ 36us) service_pool.cc:224] Handling call
0505 07:54:21.067277 (+10009324us) ts_tablet_manager.cc:1151] Deleting tablet d7e0c8ad716e443588d4ae3aa28ed4bf
0505 07:54:21.067286 (+ 9us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:21.102313 (+ 35027us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.116644 (+ 14331us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.118147 (+ 1503us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":10009201}
I20260505 07:54:21.118791 1315 catalog_manager.cc:5002] TS eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): tablet d7e0c8ad716e443588d4ae3aa28ed4bf (table default.Table_314825f5a7f644dba828c34f3d2d740a [id=056371ab05fc41c78f31a8ac6887ecb5]) successfully deleted
I20260505 07:54:21.120000 1318 catalog_manager.cc:5002] TS d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): tablet dd3fc8a9d9f04636b1f31bb88411c1da (table default.Table_bdffe9e44ae74a94ad23b5e0a6ec737f [id=c19bd0ac6f2f4ea5b4af153c02b6b0d3]) successfully deleted
I20260505 07:54:21.120054 991 leader_election.cc:304] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters: eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:21.120289 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 9926e569032144ec9d3148f4561333ab (table default.Table_3f9462e44d2b4d138eb0510c4a571080 [id=ed71bc1104b5423caedd9268977cc018]). Expected version 1 got 0
I20260505 07:54:21.120357 3483 raft_consensus.cc:2804] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260505 07:54:21.120445 3483 raft_consensus.cc:697] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [term 1 LEADER]: Becoming Leader. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Running, Role: LEADER
I20260505 07:54:21.120651 3483 consensus_queue.cc:237] T 676e2109e59749b58931d597ce1beb9c P d63a3e535f8e4dd4a88e447253405b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.120954 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 823cd847ae2b496ca7e99998297ba2fe (table default.Table_d8d1ac5a03af445490dd8c36db28f4be [id=eb677b61f4ea498aaf962c74b7c7d5f3]). Expected version 2 got 0
I20260505 07:54:21.122116 2268 ts_tablet_manager.cc:1916] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:21.123080 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet e38f248556bd4e49b06da72a1cbd7d8f (table default.Table_5c2b133923064d5eb9eabc21fb62f098 [id=12443115e63e4428a9cb798980fcbf5b]). Expected version 1 got 0
I20260505 07:54:21.124229 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet a54d94d8a888483681858398f667a370 (table default.Table_db5c1accbc4540428bd12b85b71b9e76 [id=c11ff0fc9f444a5a83a2df3d9d7d0a0d]). Expected version 3 got 0
I20260505 07:54:21.124768 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 27ab0ebc13f349f38cf86b9643700ae1 (table default.Table_5e6bc85381a74e2491f9cb19a284db9c [id=5d0d6b47936a454c9a93cc8af031807b]). Expected version 1 got 0
I20260505 07:54:21.128427 745 tablet_service.cc:1511] Processing CreateTablet for tablet 66d19dc2258b45468ae4b6ed8b22407e (DEFAULT_TABLE table=default.Table_26167dd500974094b3adfce3f4247587 [id=f4fa96cfb8894d108cb124ef1c96206e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260505 07:54:21.128861 745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66d19dc2258b45468ae4b6ed8b22407e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260505 07:54:21.129983 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 0ffcea5808c34be595bd870c0b0f6e10 (table default.Table_6080bec05b6c4e419d157ea221b02c7c [id=64904669614d4b51a1ffe3376c05bc1c]). Expected version 1 got 0
I20260505 07:54:21.130623 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 025c94c839484adba26945b62a217f5c (table default.Table_96e345e18b864cce8826bf32cac7ef52 [id=327b63fb4f744980a47bdce7db242629]). Expected version 1 got 0
I20260505 07:54:21.133422 3296 raft_consensus.cc:493] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:21.133569 3296 raft_consensus.cc:515] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:21.134109 3296 leader_election.cc:290] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:21.134796 775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01ebfd9a3514db89708ecc483be53eb" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:21.134697 1349 catalog_manager.cc:5671] T 7071df211bd34b86a751142bae3271b2 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:21.135316 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 994dbc9aa33248ee991dcf76018fe520 (table default.Table_87f5f5d0d4cf405d9d2621b5dcae27ff [id=9f1e6089d37e4c4cafb04f3dbbffa8f6]). Expected version 1 got 0
I20260505 07:54:21.135658 2447 tablet_replica.cc:333] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c: stopping tablet replica
I20260505 07:54:21.137171 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 490483562d8f4c2daa7eaa333a9f3a0c (table default.Table_3acf7c3eb13d473a9d952098c7524753 [id=fe7ee6e8118840e8a9f70a007d80c2b3]). Expected version 1 got 0
I20260505 07:54:21.137970 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 06d87dc359cb48c2b5a6f5008f33427e (table default.Table_89f2ae4a7f434e59a9039f98ec184e0b [id=2528290a3f144a57b3867c9e7ed299ae]). Expected version 1 got 0
W20260505 07:54:21.139962 854 leader_election.cc:343] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:21.141408 1047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c01ebfd9a3514db89708ecc483be53eb" candidate_uuid: "d2799f5c6b4f4336b976ca9bbb458533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d63a3e535f8e4dd4a88e447253405b77" is_pre_election: true
W20260505 07:54:21.141830 856 leader_election.cc:343] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:21.141944 856 leader_election.cc:304] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533; no voters: d63a3e535f8e4dd4a88e447253405b77, eac7e1e8caba43cdb4df80cbb80e891c
I20260505 07:54:21.142297 3296 raft_consensus.cc:2749] T c01ebfd9a3514db89708ecc483be53eb P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:21.148243 1181 tablet_bootstrap.cc:492] T d86214559d014f028acc5a9691efc9ca P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:21.148422 1181 ts_tablet_manager.cc:1403] T d86214559d014f028acc5a9691efc9ca P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.051s user 0.004s sys 0.000s
I20260505 07:54:21.148625 2447 raft_consensus.cc:2243] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260505 07:54:21.148855 2447 raft_consensus.cc:2272] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260505 07:54:21.149326 1181 raft_consensus.cc:359] T d86214559d014f028acc5a9691efc9ca P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.149428 2447 ts_tablet_manager.cc:1916] T ba616702dc6040bb805bb3ffc8a8a871 P eac7e1e8caba43cdb4df80cbb80e891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260505 07:54:21.149523 1172 tablet_bootstrap.cc:492] T 152176572c904b8296073afad0585817 P d2799f5c6b4f4336b976ca9bbb458533: No bootstrap required, opened a new log
I20260505 07:54:21.149545 1181 raft_consensus.cc:385] T d86214559d014f028acc5a9691efc9ca P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:21.149686 1172 ts_tablet_manager.cc:1403] T 152176572c904b8296073afad0585817 P d2799f5c6b4f4336b976ca9bbb458533: Time spent bootstrapping tablet: real 0.112s user 0.017s sys 0.033s
I20260505 07:54:21.149710 1181 raft_consensus.cc:740] T d86214559d014f028acc5a9691efc9ca P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:21.150813 1181 consensus_queue.cc:260] T d86214559d014f028acc5a9691efc9ca P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.150430 1172 raft_consensus.cc:359] T 152176572c904b8296073afad0585817 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:21.151149 1172 raft_consensus.cc:385] T 152176572c904b8296073afad0585817 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:21.151680 1181 ts_tablet_manager.cc:1434] T d86214559d014f028acc5a9691efc9ca P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:21.151628 1172 raft_consensus.cc:740] T 152176572c904b8296073afad0585817 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2799f5c6b4f4336b976ca9bbb458533, State: Initialized, Role: FOLLOWER
I20260505 07:54:21.151980 1181 tablet_bootstrap.cc:492] T 411025888b55405586b128d5bc0f2021 P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:21.154173 1181 tablet_bootstrap.cc:654] T 411025888b55405586b128d5bc0f2021 P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:21.166021 3366 consensus_queue.cc:1048] T 310ef5aef7e44ec5a062a01d2a175a11 P eac7e1e8caba43cdb4df80cbb80e891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260505 07:54:21.151929 1172 consensus_queue.cc:260] T 152176572c904b8296073afad0585817 P d2799f5c6b4f4336b976ca9bbb458533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:21.178704 1172 ts_tablet_manager.cc:1434] T 152176572c904b8296073afad0585817 P d2799f5c6b4f4336b976ca9bbb458533: Time spent starting tablet: real 0.029s user 0.014s sys 0.002s
I20260505 07:54:21.179016 1172 tablet_bootstrap.cc:492] T 490483562d8f4c2daa7eaa333a9f3a0c P d2799f5c6b4f4336b976ca9bbb458533: Bootstrap starting.
I20260505 07:54:21.180058 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "180157193a184add848ba67ca8042109" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c"
I20260505 07:54:21.181638 1172 tablet_bootstrap.cc:654] T 490483562d8f4c2daa7eaa333a9f3a0c P d2799f5c6b4f4336b976ca9bbb458533: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:21.186118 3489 leader_election.cc:290] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Requested vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:21.164896 991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.105.2:43839, user_credentials={real_user=slave}} blocked reactor thread for 44877.1us
I20260505 07:54:21.174378 3173 raft_consensus.cc:493] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:21.189383 3173 raft_consensus.cc:515] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.189953 3173 leader_election.cc:290] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247)
W20260505 07:54:21.180513 988 leader_election.cc:343] T 180157193a184add848ba67ca8042109 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:21.192067 907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839b963b04b24cf8af8d358a69af4f39" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:21.192256 907 raft_consensus.cc:2468] T 839b963b04b24cf8af8d358a69af4f39 P d2799f5c6b4f4336b976ca9bbb458533 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d63a3e535f8e4dd4a88e447253405b77 in term 0.
I20260505 07:54:21.195520 3366 raft_consensus.cc:493] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260505 07:54:21.195698 3366 raft_consensus.cc:515] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } }
I20260505 07:54:21.196221 3366 leader_election.cc:290] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839), d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069)
I20260505 07:54:21.196995 912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b48208e68247e78c12f0fec7ba07dc" candidate_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2799f5c6b4f4336b976ca9bbb458533" is_pre_election: true
I20260505 07:54:21.197240 912 raft_consensus.cc:2393] T 21b48208e68247e78c12f0fec7ba07dc P d2799f5c6b4f4336b976ca9bbb458533 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eac7e1e8caba43cdb4df80cbb80e891c in current term 1: Already voted for candidate d63a3e535f8e4dd4a88e447253405b77 in this term.
W20260505 07:54:21.197978 717 leader_election.cc:343] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d63a3e535f8e4dd4a88e447253405b77 (127.0.105.3:42069): Not found: Tablet not found: 21b48208e68247e78c12f0fec7ba07dc
I20260505 07:54:21.198079 717 leader_election.cc:304] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eac7e1e8caba43cdb4df80cbb80e891c; no voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77
I20260505 07:54:21.198477 3366 raft_consensus.cc:3060] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:21.199113 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet aacf715cb07d4fe9b2c57caea594ab87 (table default.Table_1c26ba8978f749a38cfdb26ff048283c [id=37b0844f588e413f99413fa009c1ef0c]). Expected version 2 got 0
I20260505 07:54:21.200052 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet d86214559d014f028acc5a9691efc9ca (table default.Table_90c98f62112f465294c60c56f6835a93 [id=77bd7e34452244009c813333c4c32ceb]). Expected version 2 got 0
I20260505 07:54:21.200630 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 68e608ac01ef4a88906e585bd7ee25ac (table default.Table_65d14038399640c284b8382aa80e344f [id=fc9aba6852ee42cc83697f86e0e69c5e]). Expected version 2 got 0
I20260505 07:54:21.200953 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 09019fdf60374b11bf3427f21a55b5e0 (table default.Table_e7ee181e7c8e4cd9a155996757c94903 [id=09bd9815349347ab9b75c4242c354c57]). Expected version 3 got 0
I20260505 07:54:21.201016 3366 raft_consensus.cc:2749] T 21b48208e68247e78c12f0fec7ba07dc P eac7e1e8caba43cdb4df80cbb80e891c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260505 07:54:21.201261 1349 catalog_manager.cc:5671] T 49d7b9f5b6f74f8ea070f8ea7e545df7 P d2799f5c6b4f4336b976ca9bbb458533 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2). New cstate: current_term: 1 leader_uuid: "d2799f5c6b4f4336b976ca9bbb458533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } health_report { overall_health: UNKNOWN } } }
I20260505 07:54:21.202100 1349 catalog_manager.cc:5763] TS d2799f5c6b4f4336b976ca9bbb458533 (127.0.105.2:43839) does not have the latest schema for tablet 1506cd31fe364e328867bbf70e36845e (table default.Table_eee3bd15017341fca226f671948e94e9 [id=c7cad0401e40485e9ed03160b9bed704]). Expected version 1 got 0
I20260505 07:54:21.202440 991 leader_election.cc:304] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2799f5c6b4f4336b976ca9bbb458533, d63a3e535f8e4dd4a88e447253405b77; no voters:
I20260505 07:54:21.202777 3261 raft_consensus.cc:2804] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260505 07:54:21.202908 3261 raft_consensus.cc:493] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260505 07:54:21.203029 3261 raft_consensus.cc:3060] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Advancing to term 1
I20260505 07:54:21.204691 776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "839b963b04b24cf8af8d358a69af4f39" candidate_uuid: "d63a3e535f8e4dd4a88e447253405b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" is_pre_election: true
I20260505 07:54:21.204898 2268 ts_tablet_manager.cc:1929] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:21.205022 2268 log.cc:1199] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-2/wal/wals/01431ad112e8445487823efe8feb0dbe
W20260505 07:54:21.205470 988 leader_election.cc:343] T 839b963b04b24cf8af8d358a69af4f39 P d63a3e535f8e4dd4a88e447253405b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eac7e1e8caba43cdb4df80cbb80e891c (127.0.105.1:36247): Illegal state: must be running to vote when last-logged opid is not known
I20260505 07:54:21.216385 1176 tablet_bootstrap.cc:492] T b8a85b65b1904bd6b793af48f71ed726 P eac7e1e8caba43cdb4df80cbb80e891c: No bootstrap required, opened a new log
I20260505 07:54:21.216531 1176 ts_tablet_manager.cc:1403] T b8a85b65b1904bd6b793af48f71ed726 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent bootstrapping tablet: real 0.102s user 0.012s sys 0.027s
I20260505 07:54:21.217175 1181 tablet_bootstrap.cc:492] T 411025888b55405586b128d5bc0f2021 P d63a3e535f8e4dd4a88e447253405b77: No bootstrap required, opened a new log
I20260505 07:54:21.217351 1181 ts_tablet_manager.cc:1403] T 411025888b55405586b128d5bc0f2021 P d63a3e535f8e4dd4a88e447253405b77: Time spent bootstrapping tablet: real 0.065s user 0.002s sys 0.002s
I20260505 07:54:21.217259 1176 raft_consensus.cc:359] T b8a85b65b1904bd6b793af48f71ed726 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.217419 1176 raft_consensus.cc:385] T b8a85b65b1904bd6b793af48f71ed726 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:21.217481 1176 raft_consensus.cc:740] T b8a85b65b1904bd6b793af48f71ed726 P eac7e1e8caba43cdb4df80cbb80e891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7e1e8caba43cdb4df80cbb80e891c, State: Initialized, Role: FOLLOWER
I20260505 07:54:21.217736 1176 consensus_queue.cc:260] T b8a85b65b1904bd6b793af48f71ed726 P eac7e1e8caba43cdb4df80cbb80e891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.218246 1176 ts_tablet_manager.cc:1434] T b8a85b65b1904bd6b793af48f71ed726 P eac7e1e8caba43cdb4df80cbb80e891c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260505 07:54:21.218236 1181 raft_consensus.cc:359] T 411025888b55405586b128d5bc0f2021 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.218477 1181 raft_consensus.cc:385] T 411025888b55405586b128d5bc0f2021 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260505 07:54:21.218590 1176 tablet_bootstrap.cc:492] T b12d2d2b866348d781209fa787da7d7e P eac7e1e8caba43cdb4df80cbb80e891c: Bootstrap starting.
I20260505 07:54:21.218575 1181 raft_consensus.cc:740] T 411025888b55405586b128d5bc0f2021 P d63a3e535f8e4dd4a88e447253405b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63a3e535f8e4dd4a88e447253405b77, State: Initialized, Role: FOLLOWER
I20260505 07:54:21.219092 1181 consensus_queue.cc:260] T 411025888b55405586b128d5bc0f2021 P d63a3e535f8e4dd4a88e447253405b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d63a3e535f8e4dd4a88e447253405b77" member_type: VOTER last_known_addr { host: "127.0.105.3" port: 42069 } } peers { permanent_uuid: "d2799f5c6b4f4336b976ca9bbb458533" member_type: VOTER last_known_addr { host: "127.0.105.2" port: 43839 } } peers { permanent_uuid: "eac7e1e8caba43cdb4df80cbb80e891c" member_type: VOTER last_known_addr { host: "127.0.105.1" port: 36247 } }
I20260505 07:54:21.220105 1181 ts_tablet_manager.cc:1434] T 411025888b55405586b128d5bc0f2021 P d63a3e535f8e4dd4a88e447253405b77: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260505 07:54:21.220415 1181 tablet_bootstrap.cc:492] T 99850198f4d04607b52953342daa753e P d63a3e535f8e4dd4a88e447253405b77: Bootstrap starting.
I20260505 07:54:21.220582 1176 tablet_bootstrap.cc:654] T b12d2d2b866348d781209fa787da7d7e P eac7e1e8caba43cdb4df80cbb80e891c: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:21.221856 2276 ts_tablet_manager.cc:1929] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260505 07:54:21.221971 2276 log.cc:1199] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533: Deleting WAL directory at /tmp/dist-test-tasklSS7ae/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1777967629171066-420-0/minicluster-data/ts-1/wal/wals/70051011024f4971bd0166d335d32393
I20260505 07:54:21.222370 2276 ts_tablet_manager.cc:1950] T 70051011024f4971bd0166d335d32393 P d2799f5c6b4f4336b976ca9bbb458533: Deleting consensus metadata
I20260505 07:54:21.222386 1181 tablet_bootstrap.cc:654] T 99850198f4d04607b52953342daa753e P d63a3e535f8e4dd4a88e447253405b77: Neither blocks nor log segments found. Creating new log.
I20260505 07:54:21.223098 2276 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53998 (request call id 278) took 6045 ms. Trace:
I20260505 07:54:21.223820 2268 ts_tablet_manager.cc:1950] T 01431ad112e8445487823efe8feb0dbe P d63a3e535f8e4dd4a88e447253405b77: Deleting consensus metadata
I20260505 07:54:21.225121 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 434) took 3135 ms. Trace:
I20260505 07:54:21.225235 2268 rpcz_store.cc:276] 0505 07:54:18.089724 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.110292 (+ 20568us) service_pool.cc:224] Handling call
0505 07:54:21.112530 (+3002238us) ts_tablet_manager.cc:1151] Deleting tablet 01431ad112e8445487823efe8feb0dbe
0505 07:54:21.112540 (+ 10us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:21.132940 (+ 20400us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.204844 (+ 71904us) tablet_metadata.cc:665] Metadata flushed
0505 07:54:21.225098 (+ 20254us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3002052}
I20260505 07:54:21.225697 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 439) took 2941 ms. Trace:
I20260505 07:54:21.225785 2268 rpcz_store.cc:276] 0505 07:54:18.284225 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.284376 (+ 151us) service_pool.cc:224] Handling call
0505 07:54:21.225569 (+2941193us) ts_tablet_manager.cc:1151] Deleting tablet 5b7a1da5cbc94df9b3b697251087b59a
0505 07:54:21.225582 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:21.225682 (+ 100us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2941019}
I20260505 07:54:21.226189 2268 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:47472 (request call id 440) took 2941 ms. Trace:
I20260505 07:54:21.226266 2268 rpcz_store.cc:276] 0505 07:54:18.284690 (+ 0us) service_pool.cc:167] Inserting onto call queue
0505 07:54:18.284946 (+ 256us) service_pool.cc:224] Handling call
0505 07:54:21.226065 (+2941119us) ts_tablet_manager.cc:1151] Deleting tablet 794256584111488aa433596dc43db10c
0505 07:54:21.226078 (+ 13us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0505 07:54:21.226174 (+ 96us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us"